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Introduction 

The  goal  of  this  project  is  to  develop  an  Evidence-based  Clinical  Decision  Support  System  (CDSS- 
EBM)  available  at  the  point  of  care  which  will  improve  prognostication  of  life  expectancy  of 
terminally  ill  patients  and  facilitate  the  hospice  referral  process.  In  addition,  the  CDSS-EBM  will 
be  expanded  with  an  evidence  based  pain  management  module  (EB-PMM)  to  assist  physicians 
managing  patients  with  pain. 

Body: 

Key  research-related  accomplishments  (since  the  submission  of  previous  annual  progress 
report): 

Currently,  the  study  is  being  conducted  at  the  Moffitt  Cancer  Center  (MCC)  and  Tampa  General 
Hospital  (TGH). 

We  have  submitted  the  no  cost  extension  application. 

Our  progress  regarding  the  task  outlined  in  the  statement  of  work  is  as  follows: 

Task  5:  Implementation  of  EBM-CDSS  to  calculate  life  expectancy  and  referral  decision 
thresholds  using  decision  curve  analysis  (DCA)  and  acceptable  regret  (ARg)  models 

•  At  the  TGH  site  we  have  screened  636  participants  for  eligibility  and  enrolled  a  total  of 
88  study  participants.  At  the  MCC  site  we  have  screened  171  individuals  and  have 
enrolled  39  study  participants. 

•  We  identified  research  associates  (one  at  each  study  site)  to  serve  as  a  back-up  for  our 
current  research  associates  to  interview  patients,  collect  data  and  manage  the  study 
related  activities  at  our  MCC  and  TGH  sites. 

•  We  completed  the  training  of  the  new  research  associates  regarding  EBM-CDSS 
software  and  data  collection  procedures.  This  step  was  essential  to  continue  the 
prospective  phase  of  our  study  at  our  study  sites  i.e.  to  enable  continuing  enrollment 
patients  when  our  RAs  are  on  vacation,  or  call  in  sick  etc. 

•  As  a  result,  we  submitted  an  amendment  to  the  University  of  South  Florida  (USF) 
Institutional  Review  Board  to  request  to  include  these  individuals  to  the  study  and 
obtained  the  approval  for  this  request. 

•  We  invested  significant  amount  of  time  in  training  the  new  research  associates  in  using 
the  EBM-CDSS  software  and  fine  tuning  their  interviewing  skills. 

•  We  designed  the  Spanish  version  of  the  informed  consent  forms  for  our  study.  A 
certified  translator  translated  the  original  English  language  informed  consent  forms  into 
Spanish  language.  This  lends  credibility  to  the  translated  version  of  the  consents 
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ensuring  that  the  consents  have  the  same  content  as  the  English  language.  We  also 
submitted  the  official  certificate  of  translation.  We  obtained  an  approval  for  these 
translated  documents  from  the  USF  Institutional  Review  Board. 

•  We  conducted  a  number  of  meetings  with  MCC  and  TGH  physicians  and  presented  our 
research  study.  These  meeting  were  fruitful;  especially  in  establishing  trust  and  working 
relationship  with  MCC  physicians  which  was  also  evident  by  inclusion  of  our  study  as 
part  of  the  MCC  clinical  trials  flow  software.  This  will  boost  the  number  of  patients 
referred  by  MCC  physicians  for  participation  in  our  study.  Specifically,  the  PI  and 
research  coordinators  have  given  a  number  of  presentations  to  the  referring  physicians, 
social  workers,  nurses  and  staff  at  Moffitt  and  TGH  to  educate  them  on  the  study  as  well 
as  ways  that  they  can  help  with  the  referral  process.  This  helped  the  awareness  with  the 
study  in  the  Thoracic  Clinic,  Head  and  Neck  Clinic  and  the  Senior  Adult  Oncology 
Program  at  MCC  as  well  as  Palliative  Care  at  TGH. 

•  We  have  met  with  the  number  of  the  key  physicians  and  their  teams  (and  continue  to 
meet)  to  explain  the  purpose  of  the  study.  This  will  be  continuing  effort  on  our  part  as 
the  study  cannot  succeed  without  adequate  referrals  from  the  physicians  at  MCC  and 
TGH. 

•  We  have  used  various  strategies  to  raise  awareness  of  our  study  to  the  referring 
physicians  from  the  various  specialties  at  MCC  and  TGH  in  order  to  improve  enrollment 
of  the  patients  in  the  prospective  phase  of  our  study.  As  a  result  recruitment  has  been 
expanded  to  include  the  Head  and  Neck  cancers  program  in  addition  to  senior  adult 
oncology,  sarcoma,  malignant  hematology,  gastrointestinal,  and  thoracic  oncology 
programs. 

•  We  have  refined  our  Evidence-based  Chronic  Pain  Management  Module  to  complement 
the  CDSS-EBM.  Our  objective  is  to  develop  a  reliable  dosage  conversion  system  as  well 
as  a  knowledge  base  for  each  available  pain  medication.  We  have  also  incorporated 
evidence  profiles  for  each  drug  to  support  the  decision  making  using  our  pain 
management  module. 

•  We  have  designed  and  developed  two  different  versions  of  the  pain  module.  The  first 
version  is  accessible  through  the  web  by  any  computer  with  internet  access  and  the 
second  is  optimized  to  be  used  on  mobile  iPad  devices.  Both  versions  of  the  pain 
module  are  guided  by  the  National  Comprehensive  Cancer  Network  (NCCN)  guidelines 
and  include  medication  evidence  profiles  that  we  have  generated.  In  addition,  we  have 
created  test  cases  that  will  be  used  to  test  our  applications  first  in  house  and  then  with 
our  collaborators  before  full  deployment. 

•  We  have  also  created  a  survey  to  test  usefulness  of  EB-PMM  its  users.  The  system  went 
through  the  final  programming  phase  and  it  will  be  first  tested  internally  and  then  in  the 
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clinic  in  the  prospective  phase  of  the  study.  We  have  also  created  the  users  manual  for 
the  EB-PMM. 

•  The  iOS  (Ipad)  based  version  of  our  EBM-PMM  is  designed  to  assist  physicians  manage 
pain  in  adult  cancer  patients.  The  application  includes  the  following  functionalities: 

o  Pain  screening  with  standardized  pain  rating  scale  used  to  determine  the 
patient's  level  of  pain; 

o  Selection  of  the  appropriate  medication  based  on  to  the  levels  of  pain,  type  of 
patient  (opioid  naive  or  opioid  tolerant)  and  patient's  preferences; 

o  Calculation  of  total  daily  dose  and  single  dose  according  to  the  medication 
presentation/concentration. 

o  Conversion  or  rotation  from  one  opioid  to  another  opioid  medication. 

o  Prescription  generation. 

We  plan  to  test  the  usability  and  functionality  of  the  application  in  our  clinical  sites. 

•  Published  three  manuscripts  in  peer-reviewed  journals  and  one  meeting  abstract  was 
accepted  for  an  oral  presentation  at  a  prestigious  national  meeting. 

Reportable  outcomes 

1.  Publications  so  far: 

•  Eleazar  Gil-Herrera,  AN  Yalcin,  Athanasios  Tsalatsanis,  Laura  E.  Barnes  and 
Benjamin  Djulbegovic,  "Towards  a  Classification  Model  to  Identify  Hospice 
Candidates  in  Terminally  III  Patients",  to  appear  in  the  Proceedings  of  the 
Annual  International  Conference  of  the  IEEE  Engineering  in  Medicine  and  Biology 
Society,  2012 

•  Miladinovic  B,  Kumar  A,  Mhaskar  R,  Kim  S,  Schonwetter  R,  et  al.  (2012)  A  Flexible 
Alternative  to  the  Cox  Proportional  Hazards  Model  for  Assessing  the  Prognostic 
Accuracy  of  Hospice  Patient  Survival.  PLoS  ONE  7(10):  e47804. 

doi:  10. 1371/journal,  pone. 0047804 

•  A.  Tsalatsanis,  I.  Hozo,  A.  Vickers,  B.  Djulbegovic,  "A  regret  theory  approach  to 
decision  curve  analysis:  A  novel  method  for  eliciting  decision  makers' 
preferences  and  decision-making",  BMC  Medical  Informatics  and  Decision 
Making  2010, 10:51  (16  September  2010) 

•  A.  Tsalatsanis,  L.  Barnes,  I.  Hozo,  B.  Djulbegovic,  "Extensions  to  Regret-based 
Decision  Curve  Analysis:  An  Application  to  hospice  referral  for  terminal  patients", 
BMC  Medical  Informatics  and  Decision  Making  2011,  11:77  (23  December  2011) 

•  E.  Gil-Herrera,  A.  Yalcin,  A.  Tsalatsanis,  L.  Barnes,  B.  Djulbegovic,  "Rough  set 
theory  based  prognostication  of  life  expectancy  for  terminally  ill  patients". 
Proceedings  of  the  IEEE  EMBC  2011 
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•  Mhaskar  R,  Miladinovic  B,  Tsalatsanis  A,  Mbah  A,  Kumar  A,  Kim  S,  Schonwetter  R, 
Djulbegovic  B.  External  Validation  of  Prognostic  Models  in  Terminally  III  Patients. 
In:  Hematology  ASo,  editor.  American  Society  of  Hematology  Annual  Conference; 
San  Diego,  California,  2011 

•  Wao  H,  Mhaskar  R,  Kumar  A,  Miladinovic  B,  Djulbegovic  B.  Survival  of  patients 
with  non-small  cell  lung  cancer  without  treatment:  a  systematic  review  and 
meta-analysis.  Systematic  reviews.  2013;  2(1):  10. 

•  Miladinovic  B,  Mhaskar  R,  Kumar  A,  Kim  S,  Schonwetter  R,  Djulbegovic  B. 

External  validation  of  a  web-based  prognostic  tool  for  predicting  survival  in 
patients  in  hospice  care.  Journal  of  Palliative  Care,  2013. 


2.  Journal  publications  since  last  progress  report:  (appendix  1) 

•  Hernandez  JM,  Tsalatsanis  A,  Humphries  LA,  Miladinovic  B,  Djulbegovic  B, 
Velanovich  V.  Defining  optimum  treatment  of  patients  with  pancreatic 
adenocarcinoma  using  regret-based  decision  curve  analysis.  Annals  of  surgery. 
2014;259(6):1208-14.  Epub  2013/10/31 

•  E.  Gil-Herrera,  A.  Tsalatsanis,  A.  Kumar,  R.  Mhaskar,  A.  Yalcin,  B.  Djulbegovic, 
"Identifying  homogeneous  subgroups  for  individual  patient  meta-analysis  based  on 
rough  set  theory",  in  proceedings  of  the  IEEE  Annual  International  Conference  of 
EMBS,  Chicago,  Illinois,  2014. 

•  A.  Tsalatsanis,  I.  Hozo,  B.  Djulbegovic,  "Empirical  evaluation  of  the  acceptable  regret 
model  of  medical  decision  making"  (oral  presentation).  Annual  Meeting  of  Society 
for  Medical  Decision  Making,  Miami,  Florida,  Oct  18-22,  2014. 

•  Pathak  EB,  Wieten  S,  Djulbegovic  B.  From  hospice  to  hospital:  short-term  follow-up 
study  of  hospice  patient  outcomes  in  a  US  acute  care  hospital  surveillance  system. 
BMJ  open.  2014;  4(7):e005196.  Epub  2014/07/24. 


Conclusions 

We  have  already  completed  the  majority  of  tasks  described  in  the  statement  of  work.  We 
believe  that  we  have  closely  followed  the  grant's  timeline  where  we  could  control  the  work 
process.  At  this  point,  we  are  focusing  on  enhancing  enrollment  of  patients  in  our  study  and 
testing  our  Pain  Decision  Support  System.  The  adequate  patient  accrual  and  empirical  testing 
of  Pain  Decision  Support  is  a  key  to  the  success  of  this  project.  We  estimate  that  we  would  need 
to  accrue  at  least  50-60  more  patients  for  reliable  statistical  analysis.  To  accomplish  this,  we 
would  need  another  year  to  complete  the  project.  We  have  applied  for  NCE  (no  cost  extension 
to  allow  us  to  complete  the  project  and  achieve  sound  scientific  goals  of  the  study).  PI  will 
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continue  to  carefully  monitor  the  "situation  on  the  ground"  and  further  allocate  distribution  of 
the  effort  among  the  faculty  and  the  staff  from  the  available  grant  support  to  match  the  stated 
goals  of  our  application. 


Next  Steps 

•  Our  immediate  and  most  important  next  step  is  to  continue  enrollment  of  patients  in 
the  prospective  phase  of  the  study.  This  requires  tackling  and  coordinating  multiple 
logistical,  regulatory  and  administrative  issues,  which  so  far  we  have  been  successfully 
addressing. 

•  We  will  continue  to  work  very  closely  with  TGH  palliative  team  and  team  of  co¬ 
investigators  from  MCC  to  accomplish  the  goals  of  the  study. 

•  We  will  maintain  the  quality  assurance  and  oversight  necessary  for  successful  execution 
of  the  study. 

•  The  key  next  steps  for  the  Decision  Support  System  include: 

o  Generation  of  evidence  tables  regarding  treatment  of  each  of  the  conditions 
identified  in  our  prediction  model. 

o  Incorporation  of  the  generated  evidence  in  the  calculations  of  survival 
probabilities  that  is  expected  to  lead  to  better  informed  decisions, 
o  Incorporation  of  the  pain  module  into  the  main  decision  support  system, 
o  Design  and  development  of  the  mobile  version  of  the  decision  support  system 
that  will  run  on  iPad  devices. 

•  Key  steps  for  our  evidence  based  pain  management  module  include: 

o  Updates  on  the  workflows  based  on  latest  NCCN  guidelines 
o  Generation  of  patient  profiles 

o  Testing  cycle  that  includes  an  iterative  process  of  in-house  testing  and  updates 
based  on  testing  feedback 

o  General  testing  cycles  with  our  collaborators  and  updates  based  on  testing 
feedback 

o  Full  deployment 
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•  Given  accumulated  experience  to  date,  we  have  undertaken  three  systematic  reviews 
aiming  to  synthesize  existing  research  evidence  in  the  literature  to  provide  best 
evidence  for  informed  decision-making  in  the  end-of  life  setting.  We  have  finalized  the 
protocols  and  have  collected  the  relevant  studies  to  be  included  in  these  systematic 
reviews.  We  are  in  the  data  analysis  and  manuscript  writing  phase  for  all  systematic 
reviews,  shortly  described  below. 

A  Systematic  Review  of  Prognostic  Models  for  Survival  among  Patients  near  the  End  of 
Life 

The  decision  to  forgo  curative-intent  medical  treatments  and  enroll  in  hospice  is  a  difficult 
choice  faced  by  terminally-ill  patients  and  their  families.  There  are  a  number  of  prognostic 
models  that  have  been  developed  to  help  clinicians  counsel  their  patients  about  the  best 
course  of  action,  based  on  calculations  of  expected  life  expectancy  and/or  survival 
probabilities.  So  far,  we  have  evaluated  several  of  the  most  promising  models.  As  reported 
before,  some  of  these  models  did  not  perform  well  in  our  hands,  while  more  promising 
ones  were  elected  for  testing  in  a  prospective  phase.  In  the  meantime,  several  new  models 
have  been  published  in  the  literatures  that  look  promising.  Before  incorporating  these  new 
models  in  the  prospective  phase  of  our  project,  we  have  elected  a  rigorous  systematic 
review  of  these  various  models  to  determine  which  one  deserve  further  testing  in  a 
prospective  phase  of  the  projects.  Therefore,  we  have  undertaken  a  comprehensive 
systematic  review,  focused  on  studies  which  were  published  during  2009-2014. 

A  Systematic  Review  of  Studies  of  Beliefs  about  End  of  Life  Health  Care  in  the  United 
States 

In  the  United  States  today,  there  is  a  growing  emphasis  on  "patient-centered"  health  care 
(Hickam  2013).  Proponents  define  "patient-centered"  as  health  care  which  gives  primacy  to 
patient  preferences  in  guiding  medical  decision  making  and  the  course  of  health  care. 
Patient  preferences  can  be  in  reference  to  the  process  of  care  (e.g.  medical  vs.  surgical 
management),  and/or  to  the  outcomes  of  care  (e.g.  desire  for  self-ambulation  vs.  desire  for 
complete  pain  control  after  a  procedure)  (Hickam  2013).  Our  experience  to  date  has  re¬ 
enforced  the  importance  of  understanding  the  differences  among  patients,  their  families 
and  caregivers,  and  their  health  care  providers  in  beliefs  and  attitudes  toward  end  of  life 
care.  Researchers  need  to  take  every  possible  measure  to  respect  the  values  and  wishes  of 
study  participants.  In  1997,  the  Institute  of  Medicine  published  a  landmark  report  titled 
"Approaching  Death:  Improving  Care  at  the  End  of  Life. "(Field  1997)  Since  that  time,  many 
small  studies  have  been  conducted  which  focus  on  either  patient  (e.g.  Wicher  2012),  health 
care  provider  (e.g.  Ramalingam  2013)  or  caregiver  (e.g.  Oliver  2013)  beliefs  about  end  of  life 
heath  care.  However,  there  have  been  no  comprehensive  systematic  reviews  on  this  very 
important  topic. 
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Our  team  has  undertaken  a  comprehensive  systematic  review,  focused  on  United  States 
studies  for  which  were  published  during  1998-2013,  to  capture  the  period  after  the  release 
of  the  IOM  report. 

Of  note,  our  accumulated  experience  with  decision-making  in  the  end-of-life  setting  has 
helped  us  generate  new  model  for  medical-decision  making  based  on  dual  processing 
theory,  which  takes  into  account  analytical  as  well  as  emotional,  intuitive  processes  (see 
Appendix). 

Palliative  Care  versus  Disease-Targeted  Management  for  Terminally  III  Adults:  A 
Systematic  Review  and  Meta-Analysis 

The  number  of  people  dying  as  a  result  of  terminal  illness  in  the  United  States  is  increasing 
(Seale  2000)  with  cancer  and  cardiovascular  disease  being  a  major  cause  of  death,  claiming 
580,350  and  787,931  lives  annually  (American  Cancer  Society  2013;  Go  2013).  Despite  the 
prevalence  of  'predictable'  deaths,  it  is  not  clear  if  patients  with  terminal  illness  stand  to 
benefit  or  are  harmed  by  continued  aggressive  disease-targeted  management  (DTM)  as 
opposed  to  palliative  care  (PC).  Despite  the  benefits  to  QOL  which  PC  can  provide,  many 
patients  continue  to  opt  for  DTM.  Around  40%  of  patients  with  advanced  lung  cancer 
continued  aggressive  therapy  through  the  final  month  of  life  (Temel  2008).  Overall,  around 
over  60%  of  cancer  patients  receive  aggressive  DTM  within  the  last  three  months  of  life 
(Braga  2007;  Martoni  2007;  Keam  2008;  Soh  2012).  Over  time,  the  number  of  patients 
receiving  aggressive  therapy  within  the  last  month  of  life  has  risen  (Gonsalves  2011). 

In  contrast,  some  studies  show  a  lack  of  survival  benefit  and  increased  adverse  events 
associated  with  aggressive  DTM  in  terminally  ill  patients  (von  Gruenigen  2008;  Saito  2011; 
Temel  2010).  Moreover,  PC  alone  approach  was  associated  with  longer  overall  survival 
among  patients  with  congestive  heart  failure,  lung  cancer,  and  pancreatic  cancer  (Connor 
2007).  Additionally,  greater  patient  satisfaction,  fewer  hospital  and  intensive  care 
admissions,  and  lower  overall  health  care  costs  is  associated  with  use  of  interdisciplinary  PC 
team  compared  to  standard  DTM  (Costantini  2003;  Gade  2008).  Three  prospective  studies 
showed  that  patients  in  PC,  as  opposed  to  aggressive  treatment  (Doubek  2005)  or  curative 
resection  (Smoot  2008)  or  surgery/chemoradiation  (Roh  2008)  have  a  shorter  median 
survival  time.  Providing  the  best  evidence  on  the  role  of  PC  vs.  DTM  is  one  of  the  key 
ingredients  of  our  EBM  decision-support  system.  The  "gold  standard"  methodology  to 
provide  and,  importantly,  update  such  evidence  is  to  employ  a  technique  of  systematic 
review,  and  if  applicable,  followed  by  meta-analysis.  (We  will  apply  both  traditional  meta- 
analytic  techniques  as  well  as  the  one  based  on  rough-set  theory,  which  we  recently 
developed.  The  application  of  rough  set  theory  is  directly  informed  by  our  use  of  this 
technique  in  this  project  so  far;  Appendix  2). 

To  date,  no  systematic  review  has  assessed  whether  use  of  PC  versus  DTM  in  management 
of  terminally  ill  patients  has  an  impact  on  survival  and  QOL.  While  studies  comparing  PC 
versus  DTM  exist  (Casarett  2001;  Connor  2007;  Costantini  2003;  Gade  2008;  Lewin  2005; 
Temel  2010),  we  are  not  aware  of  a  systematic  review  and  meta-analysis  of  such  studies. 
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Therefore,  we  are  conducting  a  systematic  review  to  compare  benefits  and  harms  of 
palliative  care  versus  disease-targeted  management  for  the  management  of  adults  with 
terminal  illness.  In  parallel,  we  are  continuing  to  generate  evidence  tables  from  the  ongoing 
literature  search  about  the  effects  of  DTM  in  the  end  of  life  setting.  The  results  of  this 
systematic  review  will  be  used  to  update  evidence  in  our  decision-support  system. 
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Background:  Pancreatic  adenocarcinoma  is  uniformly  fatal  without  operative  intervention.  Resection  can 
prolong  survival  in  some  patients;  however,  it  is  associated  with  significant  morbidity  and  mortality.  Regret 
theory  serves  as  a  novel  framework  linking  both  rationality  and  intuition  to  determine  the  optimal  course  for 
physicians  facing  difficult  decisions  related  to  treatment. 

Methods:  We  used  the  Cox  proportional  hazards  model  to  predict  survival  of  patients  with  pancreatic 
adenocarcinoma  and  generated  a  decision  model  using  regret-based  decision  curve  analysis,  which  integrates 
both  the  patient’s  prognosis  and  the  physician’s  preferences  expressed  in  terms  of  regret  associated  with  a 
certain  action.  A  physician’s  treatment  preferences  are  indicated  by  a  threshold  probability,  which  is  the 
probability  of  death/survival  at  which  the  physician  is  uncertain  whether  or  not  to  perform  surgery.  The 
analysis  modeled  three  possible  choices:  perform  surgery  on  all  patients,  never  perform  surgery,  and  act 
according  to  the  prediction  model. 

Results:  The  records  of  156  consecutive  patients  with  pancreatic  adenocarcinoma  were  retrospectively 
evaluated  by  a  single  surgeon  at  a  tertiary  referral  center.  Significant  independent  predictors  of  overall 
survival  included  preoperative  stage  (p=0.005,  Cl  1.19-2.27),  vitality  (p<0.001,  Cl  0.96-0.98),  daily  physical 
function  (p<0.001,  Cl  0.97-0.99)  and  pathologic  stage  (p<0.001,  Cl  3.06-16.05).  Compared  with  the 
“always  aggressive”  or  “always  passive”  surgical  treatment  strategies,  the  survival  model  was  associated 
with  the  least  amount  of  regret  for  a  wide  range  of  threshold  probabilities. 

Conclusions:  Regret-based  decision  curve  analysis  provides  a  novel  perspective  for  making  treatment- 
related  decisions  by  incorporating  the  decision  maker’s  preferences  expressed  as  his/her  estimates  of 
benefits  and  harms  associated  with  the  treatment  considered. 
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INTRODUCTION 
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mortality  for  patients  with  localized  pancreatic  adenocarcinoma,  pancreaticoduodenectomy  remains 
associated  with  significant  morbidity(l,  2).  Moreover,  long-term  survival  has  remained  unchanged  and 
persistently  elusive  for  the  vast  majority  of  patients  with  the  disease(3,  4).  Operative  extirpation,  for  which 
about  15-20%  of  patients  are  eligible,  is  undertaken  when  technically  feasible  because  it  offers  the  only 
opportunity  for  prolonged  survival,  and  because  there  are  few  alternative  treatments  -  each  of  which  has 
limited  efficacy(5).  However,  even  among  patients  undergoing  complete  tumor  extirpation  with  negative 
margins,  the  disease  recurs  in  40%  of  the  patients  within  6  months,  most  commonly  in  the  form  of  liver 
metastasis  (6).  These  patients  may  derive  little-to-no  survival  benefit  from  local  control,  while  potentially 
suffering  from  operative  morbidity(6).  Selection  of  patients  likely  to  benefit  from  aggressive  local  control  is 
therefore  particularly  important  in  the  management  of  patients  with  radiographic-localized  pancreatic 
adenocarcinoma. 

Decision  analysis  typically  defines  the  probability  of  an  event  and  provides  the  optimal  model  among 
alternative  clinical  management  strategies,  thus  maximizing  a  definable  outcome  (7,  8).  Probability  models 
based  on  diagnostic  and  prognostic  variables  have  been  utilized  to  assist  physician  decision-making 
regarding  various  treatments  and  interventions,  including  resection  for  cancer,  although  the  effectiveness  of 
the  models  remains  questionable(9-15).  The  reasons  behind  this  skepticism  include  the  probabilistic  nature 
of  these  models  that  adds  complexity  to  the  decision  process  and,  importantly,  the  reliance  of  most  of  these 
models  on  expected  utility  theory,  which  is  often  violated  during  decision  making(  16-20). 

We  recently  developed  a  decision  methodology  that  overcomes  the  limitations  of  probabilistic 
survival  models,  and  which  can  be  utilized  to  facilitate  medical  decisions  based  on  the  decision-maker 
preferences  (19,  20).  Our  methodology,  Regret-based  Decision  Curve  Analysis  or  Regret  DCA,  relies  on  the 
cognitive  emotion  of  regret  to  identify  conditions  under  which  a  physician  is  unsure  about  the  choice 
between  alternative  treatment  strategies  (19,  20).  Surgeons,  as  with  any  decision  maker,  may  experience 
regret  (defined  as  the  difference  between  the  utility  of  an  action  taken  and  utility  of  an  alternative  action)  if 
they  eventually  realize  that  a  decision  they  made  was  suboptimal,  and  that  an  alternative  fonn  of  treatment 
would  have  been  preferable  (21-27).  Regret  DCA  utilizes  this  regret  to  compute  the  threshold  probability  at 
which  the  physician  is  uncertain  about  which  treatment  strategy  to  recommend  to  his/her  patient.  In  this 
study,  we  used  Regret  DCA  to  facilitate  treatment  decisions  for  a  cohort  of  patients  with  localized, 
resectable  pancreatic  adenocarcinoma. 

The  intention  of  this  article  is  to  present  a  novel  decision  methodology  that  relies  on  regret  theory 
and  attempts  to  explain  medical  decision-making  for  surgeons  treating  patients  with  pancreatic 
adenocarcinoma.  Despite  the  fact  that  the  prediction  model  presented  has  been  well  fitted  to  our  data,  its  role 
in  this  article  is  secondary  and  its  purpose  is  to  demonstrate  how  the  regret  methodology  can  be  used  to 
evaluate  three  management  strategies:  aggressive,  passive,  or  model-based  decision  making.  In  this  context, 
we  have  demonstrated  that  the  prediction  model  performs  better  the  other  two  strategies  in  terms  of  regret. 
MATERIALS  AND  METHODS 

The  records  of  156  consecutive  patients  referred  for  surgical  consultation  from  January  2005  to  2009 
with  pancreatic  adenocarcinoma  were  retrospectively  reviewed  by  a  single  surgeon  at  a  tertiary  referral 
center.  The  diagnosis  was  confinned  by  histological  evaluation,  and  disease  stage  was  determined  by 
pathological  evaluation  of  the  resected  specimen  and  by  imaging.  All  patients  had  been  administered  the  SF- 
36  Health  Survey  to  assess  quality  of  life,  which  includes  36  statements  grouped  into  8  domains  of  quality  of 
life:  physical  functioning,  physical  role,  bodily  pain,  general  health,  vitality,  social  functioning,  emotional 
role,  and  mental  health.  The  SF-36  utilizes  a  Likert  scale  of  0  to  100,  with  higher  scores  indicating 
better/nonnal  health  or  physical  functioning.  We  previously  demonstrated  that  the  SF-36  correlates  well 
with  pathology,  survival,  stage  and  resectability  of  pancreatic  lesions  (27). 

The  distribution  for  overall  survival  was  estimated  using  the  Kaplan-Meier  Method.  Cox 
proportional  hazards  modeling  was  used  to  determine  the  effect  on  survival  of  the  following  12  covariates, 
including  those  described  by  SF-36:  age,  gender,  stage,  adjuvant  therapy,  physical  functioning,  role- 
physical,  role-emotional,  bodily  pain,  pretreatment  vitality,  mental  health,  social  functioning  and  general 


Downloaded  from  bmjopen.bmj.com  on  October  20,  2014  -  Published  by  group.bmj.com 


health.  Additional  covariates  such  as  tumor  characteristics  (lymphovascular  invasion,  perineural  invasion, 


etc.")  could  potentially  influence  the  output  of  the  Cox  model,  however,  this  information  is  typically 
unknown  to  the  surgeon  a  priori.  Furthermore,  such  covariates  were  not  included  in  the  analysis  since  our 
dataset  was  originally  constructed  based  on  the  methods  and  protocols  designed  for  a  study  (28)  focusing  on 
the  quality  of  life,  pathology,  resectability  and  survival  in  patients  with  pancreatic  lesions.  The  model  was 
created  using  stepwise  elimination  on  all  variables  (p<  0.15  to  enter,  and  p<  0.20  to  stay).  The  proportional 
hazards  assumption  was  examined  using  Schoenfeld  residuals.  The  importance  of  each  variable  and  the 
discriminative  ability  of  the  Cox  model  was  examined  using  Royston-Sauerbrei’s  discrimination  statistic  D 
and  explained  variation  R  D  (29).  All  continuous  variables  were  centered  about  the  mean.  All  analyses  were 
performed  using  STATA  (30). 

To  derive  the  optimal  treatment  strategy,  we  then  utilized  the  Regret-based  Decision  Curve  Analysis 
methodology  ( Regret  DCA)(  19,  20).  Regret  DCA  employs  the  decision  maker’s  feeling  of  regret  to  compute 
the  threshold  probability  at  which  he/she  is  uncertain  about  alternative  actions,  e.g.,  to  operate  or  not  to 
operate.  In  considering  decisions  for  patients  with  pancreatic  adenocarcinoma,  we  considered  survival  less 
than  7  months  from  the  time  of  tumor  extirpation  as  being  unlikely  to  have  imparted  a  survival  advantage, 
and  therefore  unnecessary  based  upon  median  survival  of  patients  with  locally  advanced,  non-metastatic 
disease  (31).  Based  on  this  assumption,  we  formulated  a  decision  model  that  compares  an  individual 
patient’s  prognosis  with  the  threshold  probability  at  which  the  surgeon  would  be  indifferent  about 
recommending  surgery. 

Typically,  decision  theory  suggests  that  a  person  should  be  treated  if  the  probability  of  an  event  (i.e. 
the  patient  develops  a  disease;  the  patient  dies;  the  patient  survives  longer  than  a  predefined  timeframe,  etc.) 
is  greater  than  or  equal  to  a  threshold  probability  (7,  8,  32).  In  this  paper,  we  sought  to  treat  the  patients  who 
were  likely  to  survive  longer  than  7  months  from  the  time  of  their  resection.  Therefore,  the  convention  used 

is:  if  the  patient’s  probability  of  surviving  7  months  is  greater  than  or  equal  to  the  threshold 
probability  ( s  >  Pt),  the  surgeon  should  offer  resection.  If  the  patient’s  probability  of  survival  is  less 
than  the  threshold  probability  (s  <  Pt),  the  patient  may  be  unlikely  to  benefit  substantially  from 
surgery  and  the  surgeon  should  not  recommend  resection  in  favor  of  medical  alternatives. 

The  probability  of  survival  can  be  computed  for  each  patient  based  on  the  Cox  survival  model 
previously  described.  However,  the  threshold  probability  is  subject  to  each  surgeon’s  preferences  and 
clinical  practice  attitudes.  At  the  individual  level,  it  can  be  computed  as  (19,  20): 

„  .  Regret  of  omission  (i) 


1+ 


Regret  of  commission 


We  define  “ regret  of  omission'"  as  the  regret  felt  by  a  surgeon  who  withheld  necessary  surgery  from  a  patient 
who  may  have  benefited  from  that  resection  (patients  with  localized  disease  who  lived  longer  than  7 
months).  Conversely,  “ regret  of  commission ”  is  the  regret  felt  by  a  surgeon  who  performed  an  unnecessary 
surgery  on  a  patient  who  derived  no  benefit  from  that  operation  (e.g.  the  patient  died  as  a  result  of  the 
procedure  or  died  within  7  months  from  the  time  of  resection).  Both  regret  values  can  be  determined  using 
the  Dual  Visual  Analogue  Scales  (DVAs)  (Figure  1)  (19,  20).  Formally,  regret  can  be  expressed  as  the 
difference  between  the  utility  of  the  outcome  of  an  action  taken  and  the  utility  of  the  outcome  of  the  action 
that,  in  retrospect,  should  have  been  taken  (21-27).  Commonly  used  techniques  for  estimating  utility,  and 
therefore  decision  maker  preferences,  such  as  standard  gamble  and  time  trade-off  are  time  consuming, 
cognitively  complex  and  have  been  shown  to  lead  to  biased  estimates  of  people’s  preferences  (33-35). 
Instead,  in  this  paper,  we  use  the  Dual  Visual  Analogue  Scales  (DVAs)  to  estimate  directly  the  values  of 
regret  of  commission  and  omission(19,  20).  The  DVAs  comprise  two  100-point  scales,  each  anchored  to  no 
regret  and  maximum  regret.  One  of  the  scales  is  used  to  elicit  regret  of  omission  and  the  other  to  elicit  regret 
of  commission  (Figure  1). 

After  computing  the  surgeon’s  threshold  probability,  the  clinical  question  regarding  treatment  for 
patients  with  pancreatic  adenocarcinoma  can  be  broken  down  into  three  strategies:  1.  surgeons  can  stay 
passive  and  allow  the  disease  to  run  its  course,  2.  surgeons  can  be  aggressive  and  recommend  resection  on 
all  patients,  or  3.  surgeons  can  use  prediction  model  for  guidance.  Any  of  these  strategies  may  cause  regret 
if  the  outcome  is  poor.  Under  the  Regret  DCA  methodology,  the  optimal  strategy  is  the  one  that  will  cause 
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the  least  amount  of  regret  if  that  strategy  is  proven  suboptimal.  Formally,  regret  can  be  expressed  as  the 

difference  between  the  utility  of  the  outcome  of  the  action  taken  and  the  utility  of  the  outcome  of  the  action 
that,  in  retrospect,  should  have  been  taken  (21-27).  Considering  the  decision  tree  that  describes  this  clinical 
problem  (Figure  2),  we  can  compute  the  expected  regret  associated  with  each  of  the  three  strategies  as 
follows: 

ERg  [NoSurgery]  =  (1  —  s)  *  (2) 

ERg  [Surgery]  —  s  (3) 

ERg[Model]  =  —  *  —  +  —  (4) 

The  values  of  #FP  and  #FN  correspond  to  the  number  of  false  positive  and  false  negative  results, 
respectively,  as  compared  to  the  actual  patient  outcomes  used  for  the  development  of  the  prediction  model, 
and  the  number  of  patients  in  the  dataset  is  n.  We  define  true  positive  (TP),  true  negative  (TN),  false 
positive  (FP),  and  false  negative  (FN)  results  as  follows: 

TP:  the  number  of  patients  who  will  survive  longer  than  7  months  and  for  whom  the  estimated  probability  of 
survival  is  greater  than  or  equal  to  the  threshold  probability  (i.e.,  the  patients  who  should  receive  surgery). 
TN:  the  number  of  patients  who  will  die  in  7  months  and  for  whom  the  estimated  probability  of  survival  is 
less  than  the  threshold  probability  (i.e.,  the  patients  who  should  NOT  receive  surgery). 

FP:  the  number  of  patients  who  will  die  within  7  months  and  for  whom  the  estimated  probability  of  survival 
is  greater  than  or  equal  to  the  threshold  probability  (i.e.,  the  patients  who  received  unnecessary  surgery). 

FN:  the  number  of  patients  who  will  survive  longer  than  7  months  and  for  whom  the  estimated  probability  of 
survival  is  less  than  the  threshold  probability  (i.e.,  the  number  of  patients  who  should  have  received  surgery 
but  did  not). 

As  shown  in  equations  2  and  4,  the  expected  regret  associated  with  each  strategy  is  a  function  of  the 
physician’s  threshold  probability.  To  identify  the  least  regretful  action,  the  Regret  DCA  methodology 
computes  the  expected  regret  for  a  range  of  threshold  probabilities  (0-100),  and  expected  regret  is  then 
graphed  against  the  threshold  probability  for  each  of  the  three  actions.  The  action  with  the  lowest  value  of 
expected  regret  corresponds  to  the  most  desired  action,  given  a  certain  threshold  probability. 

RESULTS 

Patient  Characteristics 

A  total  of  156  patients  with  histologically-confinned  primary  pancreatic  adenocarcinoma  were 
included.  The  mean  age  was  65.9  ±  10  years,  83%  were  stage  I  or  II,  54%  were  resected,  66%  received 
chemotherapy,  and  the  median  survival  was  18  months  (95%  Cl  12-26)  (mean  survival  was  15.7  ±  25 
months).  The  SF-36  scores  revealed  that  role-physical  and  pretreatment  vitality  had  the  lowest  scores,  and 
mental  health  had  the  highest  score  (Table  1).  The  distribution  of  overall  survival  is  presented  in  Figure  3. 
Survival  model 

Of  the  12  variables  included  in  the  dataset,  three  met  the  stepwise  inclusion  criteria  and  were  used  to 
construct  the  survival  model:  stage,  pretreatment  vitality,  and  role-physical  (daily  physical  functioning).  The 
explained  variation  of  the  fitted  model  was  R2d  =0.4  (95%  Cl:  0.27-0.52)  and  the  proportional  hazard 
assumption  were  not  violated  (P  <  0.96).  Table  2  presents  the  estimates  of  hazard  ratio  for  the  Cox 
prediction  model. 

Regret  Decision  Curve  Analysis 

We  employed  Regret  DCA  to  evaluate  the  three  management  strategies:  1.  Recommend  against 
potentially  curative  surgery  in  favor  chemotherapy  or  chemoradiotherapy;  2.  be  aggressive  and  recommend 
resection,  3.  use  the  prediction  model  as  a  decision  aid.  Figure  4  depicts  the  expected  regret  as  a  function  of 
threshold  probability  for  each  of  the  three  management  strategies.  As  shown,  the  least  regretful  strategy  for 
threshold  probabilities  greater  than  5%  is  to  utilize  the  prediction  model.  For  threshold  probabilities  between 
80-87%,  the  regret  curve  associated  with  the  prediction  model  is  subject  to  noise  (36)  that  we  attribute  to  the 
error  tenn  of  the  Cox  prediction  model.  We  assume  that  the  prediction  model  remains  the  least  regretful 
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strategy  within  the  80-87%  range  as  well.  Our  results  demonstrate  that  the  survival  model  we  describe  has 

sic 


Hypothetical  Case  Study 

A  72  year-old  female  with  diabetes  and  hypertension  has  been  diagnosed  with  pancreatic 
adenocarcinoma  after  undergoing  endoscopic  retrograde  cholangiopancreatography  (ERCP)  and  common 
bile  duct  stenting  for  obstructive  jaundice.  She  is  currently  without  pain  and  is  tolerating  a  regular  diet.  Her 
jaundice  resolved  after  the  placement  of  her  biliary  stent.  Her  CT  scan  demonstrates  a  localized  mass  in  the 
head  of  the  pancreas  without  involvement  of  the  superior  mesenteric  vein,  portal  vein,  superior  mesenteric 
artery,  or  hepatic  arteries.  The  patient  is  active  and  able  to  perform  all  activities  of  daily  living.  She 
expresses  a  strong  desire  to  spend  as  much  time  as  she  can  with  her  grandchildren. 

We  demonstrate  the  decision  process  assuming  two  types  of  hypothetical  decision  makers:  One 
surgeon  is  extremely  selective  in  offering  resection  to  patients  with  pancreatic  adenocarcinoma  (Surgeon 
#1),  and  the  second  surgeon  (Surgeon  #2)  generally  offers  resection  to  all  patients  with  radiographically- 
resectable  disease.  The  process,  depicted  in  Figure  5,  is  initiated  with  the  elicitation  of  the  surgeon’s 
preferences.  Using  the  DVAS  method  (Figure  1)  we  estimate  the  threshold  probability  as  a  function  of  regret 
of  omission  and  regret  of  commission  (equation  1).  Suppose  that  the  answers  to  the  questions  shown  in 
Figure  1  for  the  surgeons  are  as  follows: 

Surgeon  # 1 :  Regret  of  omission:  20;  regret  of  commission:  90.  Therefore,  the  threshold  probability  is  equal 

to:  81.8%  (equation  1). 

Surgeon  #2:  Regret  of  omission:  90;  regret  of  commission:  4.  Therefore,  the  threshold  probability  is  equal  to: 

4.2%. 


Based  on  the  results  of  Regret-DCA  (Figure  4),  the  optimal  and  least  regretful  strategy  for  Surgeon#  1  is  to 
use  the  prognostication  model  we  developed,  described  above.  If  the  patient’s  estimated  probability  of 
survival  is  greater  than  or  equal  to  81.8%  (the  threshold  for  Surgeon  #1)  then  the  optimal  strategy  is  to  treat 
(perform  the  operation).  If  the  probability  of  survival  is  less  than  81.8%,  then  the  optimal  strategy  is  to  offer 
alternative  treatments  (forego  resection).  Conversely,  for  Surgeon  #2,  whose  threshold  probability  is  equal  to 
4.2%,  the  optimal  and  least  regretful  strategy  is  to  offer  resection. 

As  mentioned  earlier,  the  Regret-DCA  methodology  can  also  be  used  by  the  patients  (19).  For 
completeness,  we  present  how  this  process  could  work.  The  patient  would  be  asked  questions  similar  to 
those  depicted  in  Figure  1.  We  have  previously  shown  that  patient  ratings  of  utility  scores  closely  correlate 
with  quality  of  life  after  pancreaticoduodenectomy;  moreover,  this  patient-centered  assessment  many  change 
over  time  as  quality  of  life  improves  (37). 

Regret  of  omission:  On  a  scale  of  Oto  100,  where  0  =  no  regret  and  100  =  maximum  regret  you  could  feel, 
how  would  you  rate  your  level  of  regret  if  you  did  not  have  an  operation  that  could  have  extended  your  life? 
Regret  of  commission:  On  a  scale  of  0  to  100,  where  0  =  no  regret  and  100  =  maximum  regret  you  could  feel, 
how  would  you  rate  your  level  of  regret  if  you  had  an  operation  that  did  not  extend  your  life? 


DISCUSSION 

We  describe  the  theory  and  application  of  regret  decision  curve  analysis  as  it  applies  to  surgeons  and 
to  decisions  regarding  operative  intervention  in  patients  with  pancreatic  adenocarcinoma.  To  the  best  of  our 
knowledge,  this  is  the  first  application  of  regret  DCA  to  assist  surgeons  in  decision-making  for  patients  with 
pancreatic  malignancies.  Our  approach  promotes  personalized  patient  care  by  incorporating  decision-maker 
preferences  from  the  perspective  of  regret  by  estimating  a  threshold  probability  for  a  decision  maker.  We 
believe  the  decision  regarding  resection  for  patients  with  pancreatic  adenocarcinoma  is  particularly  well 
suited  for  a  regret-based  approach  given  the  generally  fatal  prognosis  for  this  disease,  regardless  of  the 
decision  made. 

Modern  cognitive  theories  seek  to  balance  risks  and  benefits  in  the  decision-making  process  by 
taking  into  account  both  intuition  and  analytical  processes  (37).  We  believe  that  rational  decision-making 
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should  take  into  account  both  the  formal  principles  of  rationality  and  human  intuition.  We  have 
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thinking  (19,  20).  Eliciting  surgeons’  preferences  by  using  regret  is  likely  to  prove  superior  to  using 
traditional  utility  theory  because  regret  explicitly  forces  the  surgeon  to  consider  consequences  of  decisions. 
Our  method  relies  on  elicitation  of  a  threshold  probability,  which  must  be  calculated  for  every  decision 
maker.  In  other  words,  our  model  forces  surgeons  to  consider  the  possible  outcomes  of  recommending 
pancreaticoduodenectomy  rather  than  simply  recommending  resection  for  all  tumors  that  appear  resectable 
on  radiographic  imaging. 

We  argue  that  our  approach  contributes  to  the  field  of  decision-making,  but  we  acknowledge  that  it  is 
not  a  panacea.  We  do,  however,  believe  that  our  methodology  is  best  suited  for  medical  decision-making 
primarily  associated  with  trade-offs  between  quality  and  quantity  of  life.  Pancreatic  adenocarcinoma  meets 
this  criterion:  surgical  resection  may  offer  an  additional  year  of  survival,  albeit  with  the  potential  for  serious 
morbidity,  particularly  if  the  resection  is  undertaken  at  low-volume  centers  (38,  39).  For  the  fortunate  15- 
20%  of  patients  with  radiographically-localized  disease  amenable  to  resection,  the  median  survival  ranges 
from  17  to  23  months  (40).  At  high-volume  institutions  with  extensive  experience,  the  mortality  rate  is 
<3%— 5%,  but  morbidity  remains  problematic,  with  early  postoperative  complication  rates  of  ~30%-40%  (6). 
Perioperative  morbidity  and  mortality  rates  recorded  in  national  databases,  which  include  data  from  a  broad 
spectrum  of  hospitals  and  surgeons’  experiences,  report  significantly  higher  numbers  of  complications  than 
high-volume  tertiary  referral  centers  (38).  Applying  our  model  of  regret  theory  may  indirectly  motivate  each 
surgeon  to  consider  their  own  results  with  the  procedure  and  to  consider  the  support  available  within  the 
institution  where  the  procedure  is  planned  when  contemplating  the  best  course  of  action  for  each  patient, 
further  personalizing  care. 

A  significant  proportion  of  patients  undergoing  resection  develop  early  metastatic  disease  and  have 
very  limited  survival,  and  thus  derive  no  benefit  from  the  operative  intervention  (i.e.,  there  is  no  trade-off 
improvement  in  quality-of-life).  This  issue  has  been  addressed  with  the  use  of  refined  definitions  of 
borderline  resectability  and  the  use  of  neoadjuvant  therapy  (41).  Specifically,  this  minimally  effective 
chemotherapy,  which  offers  virtually  no  hope  of  eradicating  disease  and  little  if  any  therapeutic  efficacy, 
does  provide  a  “window  of  observation”,  during  which  distant  metastatic  disease  may  appear  and  thus  spare 
the  patient  unnecessary  surgery.  This  approach  may  minimize  regret  and  results  in  better  overall  survival  for 
patients  who  ultimately  undergoing  resection  (42),  but  it  has  not  been  widely  adopted  across  the  country  or 
even  across  academic  centers.  Similarly,  regret  theory  remains  severely  underutilized  in  the  healthcare  arena, 
despite  considerable  conceptual  and  empiric  interest  in  its  applicability,  and  in  the  strong  influence  of  regret 
on  physician  decision-making  (32,  43-45).  The  lack  of  incorporation  of  regret  theory  into  healthcare  delivery 
is  particularly  perplexing,  especially  considering  that  all  medical  decisions  are  accompanied  by  varying 
degrees  of  risk  and  uncertainty,  and  -  therefore  -  potential  regret.  Moreover,  recent  work  has  suggested  that 
physicians’  behavior  can  often  be  explained  by  regret  avoidance  (46),  which  further  substantiates  the  need  to 
incorporate  regret  modeling  into  healthcare  decisions. 

As  with  any  novel  theoretical  work,  our  application  of  regret  theory  to  pancreatic  adenocarcinoma 
has  limitations.  First,  we  applied  the  theory  retrospectively  with  assigned  cutoff  survival  values.  We 
assumed  maximal  regret  to  be  associated  with  operating  on  a  patient  who  died  within  the  first  seven  months 
following  resection.  Excluding  death  as  a  result  of  the  procedure  (perioperative  death),  which  is  always 
associated  with  regret,  death  within  seven  months  may  not  necessarily  be  associated  with  regret.  For 
example,  a  patient  may  have  died  of  an  unrelated  stroke  that  could  not  have  been  foreseen  prior  to  resection. 
Second,  our  approach  has  not  yet  been  empirically  tested  and  the  prediction  model  has  not  been  externally 
validated.  Third,  the  methodology,  as  presented,  is  appropriate  for  point  decision-making,  and  not 
necessarily  for  decisions  that  re-occur  over  time  -  as  frequently  happens  in  patient  care.  Finally,  we  assumed 
that  there  is  a  single  decision-maker  involved  in  the  process  where,  in  actual  practice,  a  multidisciplinary 
team  of  healthcare  providers  is  involved  in  treatment  decisions. 

In  conclusion,  we  have  described  a  novel  approach  to  surgical  decision-making  using  the  cognitive 
emotion  of  regret,  which  seeks  to  personalize  care.  The  goal  of  our  work  is  to  power  a  computerized 
decision  support  tool  to  assist  physicians  and  patients  in  making  better  medical  decisions.  We  envision  the 
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tool  to  be  shared  by  both  physician  and  patient  during  consultation,  in  which  the  physician  elicits  the 
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Table  1.  Patient  Demographics  and  SF-36  Scores.  Values  are  the  mean  ±  SEM  unless 


Male  :  Female,  n  (%) 

70:86  (45%:  55%) 

Age  (yr.), 

65.9  ±  10 

Stage:  n  (%) 

I 

61  (39%) 

II 

68  (44%) 

III 

25  (16%) 

0 

2  (1%) 

SF-36  Scores:1 

Physical  functioning 

55.2  ±31 

Role-physical 

35.5  ±44 

Role-emotional 

57.4  ±46 

Bodily  pain 

55.5  ±30 

Pretreatment  vitality 

41.8  ±  24 

Mental  health 

70.3  ±21 

Social  functioning 

60.8  ±31 

General  health 

60.7  ±22 

Patients  undergoing  resection,  n  (%) 

85  (54%) 

Patients  receiving  chemotherapy,  n  (%) 

103  (66%) 

Survival  (mo.) 

15.7  ±25 

1  SF-36  Health  Survey,  rated  from  0  to  100  on  a  Likert  scale,  with  higher  scores  indicating  better  health  or 
physical  function  (ref). 

Table  2.  Hazard  ratio  estimates  of  the  prediction  model  _ _ 


Hazard  Ratio 

P>|z| 

[95%  conf.  interval] 

Stage 

1.994865 

0.001 

1.326723-2.999486 

Pretreatment  vitality 

.9849276 

0.030 

.971512- .9985284 

Role-physical 

.9884022 

0.005 

.9803665- .9965038 
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Figure  1.  Dual  Visual  Analog  Scales.  The  DVAS  are  used  for  the  elicitation  of  the  decision  maker’s 

tlwSloTmpif'OT^bflff’^n^^tfTOSTfatfsTCpfCmfflff^CS^  "  1 "  " 


Figure  2.  Decision  model  for  performing  surgery  on  patients  suffering  from  pancreatic 
adenocarcinoma. 

5  denotes  the  probability  of  survival,  S  +  denotes  surgery  or  no  surgery,  D  +  denotes  death  or  no  death,  t/j 
are  the  utilities  associated  with  each  outcome  and  Rg  is  the  regret  associated  with  each  action.  For  example, 
Rg(S—,D+ )  is  the  regret  associated  with  not  performing  a  surgery  for  a  patient  who  died  within  7  months. 


Figure  3.0verall  survival  of  patients  with  pancreatic  adenocarcinoma  expressed  as  Kaplan-Meier 
survival  and  95%  confidence  interval  bands.  Vertical  bars  (|)  denote  censored  observations. 


Figure  4.  Regret  DCA  for  the  survival  model  constructed  using  Cox  regression  on  three  variables. 

Dashed  and  dotted  line  denotes  the  decision  to  perfonn  surgery;  solid  line  denotes  the  decision  not  to 
perform  surgery  on  any  patient;  dashed  line  denotes  the  use  of  the  survival  model  to  perfonn  surgery.  The 
optimal  strategy  is  the  action  that  results  in  the  least  amount  of  regret  in  case  it  is  proven  wrong.  For 
threshold  probabilities  of  0-5%,  the  optimal  strategy  is  to  perform  surgery  on  all  patients,  while  for  threshold 
probabilities  greater  than  5%  the  optimal  strategy  is  to  consult  the  survival  model.  For  threshold  probabilities 
between  80-87%,  the  regret  curve  associated  with  the  prediction  model  is  subject  to  noise  associated  to  the 
error  of  the  prediction  model  therefore,  we  assume  that  the  prediction  model  remains  the  least  regretful 
strategy. 


Figure  5.  Schematic  Representation  of  Decision  Model. 
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set  theory 

Abstract —  Failure  to  detect  and  manage  heterogeneity  between  clinical  trials  included  in  meta-analysis  may  lead  to  misinterpretation 
of  summary  effect  estimates.  This  may  ultimately  compromise  the  validity  of  the  results  of  the  meta-analysis.  Typically,  when 
heterogeneity  between  trials  is  detected,  researchers  use  sensitivity  or  subgroup  analysis  to  manage  it.  However,  both  methods  fail  to 
explain  why  heterogeneity  existed  in  the  first  place.  Here  we  propose  a  novel  methodology  that  relies  on  Rough  Set  Theory  (RST)  to 
detect,  explain,  and  manage  the  sources  of  heterogeneity  applicable  to  meta-analysis  performed  on  individual  patient  data  (IPD).  The 
method  exploits  the  RST  relations  of  discernibility  and  indiscernibility  to  create  homogeneous  groups  of  patients.  We  applied  our 
methodology  on  a  dataset  of  1,111  patients  enrolled  in  9  randomized  controlled  trials  studying  the  effect  of  two  transplantation 
procedures  in  the  management  of  hematologic  malignancies.  Our  method  was  able  to  create  three  subgroups  of  patients  with 
remarkably  low  statistical  heterogeneity  values  (16.8%,  0%  and  0%  respectively).  The  proposed  methodology  has  the  potential  to 
automatize  and  standardize  the  process  of  detecting  and  managing  heterogeneity  in  IPD  meta-analysis.  Future  work  involves 
investigating  the  applications  of  the  proposed  methodology  in  analyzing  treatment  effects  in  patients  belonging  to  different  risk  groups, 
which  will  ultimately  assist  in  personalized  healthcare  decision  making. 

I.  Introduction 

In  medical  research,  meta-analysis  is  used  to  obtain  pooled  estimates  of  the  treatment  effects  reported  in 
various  clinical  research  studies.  The  importance  of  meta-analysis  stems  from  the  necessity  to  combine 
research  findings  that  if  considered  separately  they  would  produce  insignificant,  non-generalizable,  and 
unavailing  results,  unfit  to  inform  medical  practice.  By  systematically  combining  findings  from  similar 
studies  it  is  possible  to  achieve  the  totality  of  evidence  necessary  to  evaluate  the  efficacy  of  an  investigated 
treatment. 

The  challenge  researchers  face  when  performing  meta-analysis  is  how  to  integrate  studies  that  present 
differences  in  the  design,  characteristics  and  reported  effects.  Such  differences  are  formally  acknowledged 
as  heterogeneity  and  they  are  defined  as  any  kind  of  variability  among  studies  [1],  Typically,  there  are  three 
types  of  heterogeneity  found  in  meta- analyses:  1.  Methodological,  which  refers  to  variability  in  the  study 
design  and  risk  of  bias  (e.g.  randomization,  allocation  concealment,  blindness  etc.)[2,  3],  2.  Clinical,  which 
refers  to  the  variability  in  the  participants,  interventions  and  outcomes  studied  (e.g.  age,  race,  disease 
severity,  disease  progression,  past  treatment  etc.)  [2,  3],  and  3.  Statistical,  which  refers  to  variability  in  the 
observed  outcomes  [1,3].  Failure  to  detect  heterogeneity  leads  to  misinterpretation  of  the  summary  effect 
estimates,  which  jeopardize  the  quality  of  the  meta-analyses  [2,  3]  and  may  produce  faulty  estimations  of  the 
effects  magnitude  [4,  5].  Both  methodological  and  clinical  heterogeneity  may  result  in  statistical 
heterogeneity  [6].  Researchers  focus  primarily  on  detecting  statistical  heterogeneity  and  subsequently  on 
determining  whether  such  heterogeneity  is  caused  due  to  methodological  or  clinical  variations  between 
studies  [1]. 

Assessing  statistical  heterogeneity  relies  on  approaches  that  involve  hypothesis  testing  [1,7-9],  such  as  the 
Chochrane’s  chi-square  (Q)  [10]  and  the  1“  measure  [9,  1 1],  Higher  values  on  these  tests  indicate  high 
heterogeneity  between  studies.  Both  chi-square  and  I  tests  focus  on  detecting  heterogeneity  yet  are  unable 
to  identify  the  specific  causes  that  underlie  heterogeneity  across  studies  [12].  The  burden  of  explaining 
heterogeneity  falls  on  the  researcher. 
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To  explore  and  explain  the  observed  heterogeneity,  meta-analysts  conduct  sensitivity  analysis,  based  on  the 

rncmoaoTogTCcKjUc^^^^^maTcs^fncRu^groupmcnysTs^asccroW^^^pccmec^nc^o^anenr 
characteristics  [3].  That  is,  the  trials  included  in  the  meta-analysis  are  grouped  according  to  pre-specified 
criteria.  In  case  of  individual  patient  data  meta-analysis  patients  are  grouped  according  to  pre-specified 
clinical  characteristics.  However,  these  pre-specified  criteria  and  clinical  characteristics  are  generated  in  an 
ad-hoc  manner  and  rely  on  the  skills  and  medical  knowledge  of  the  researcher  performing  the  meta- analysis. 
Thus,  the  results  of  meta-analysis  may  potentially  differ  depending  on  the  experience  of  the  meta-analyst. 

Subgroup  analysis  [13]  and  meta-regression[14]  are  also  applied  to  individual  patient  datasets  (IPD) 
containing  patient  characteristics  that  may  potentially  influence  the  treatment  effects.  Determining  which  set 
of  characteristics  can  be  used  to  obtain  homogeneous  groups  yields  in  a  complex  process,  where  subgroup 
analysis  and  meta-regression  have  been  found  prone  to  false  positive  results  and  ecological  bias. 

In  this  paper,  we  focus  on  meta-analyses  of  individual  patient  data  and  we  propose  a  novel 
methodology  to  identify  homogeneous  groups  of 
patients  for  managing  the  detected  heterogeneity.  Our 
approach  is  based  on  Rough  Set  Theory  (RST)  [15]  and 
has  the  potential  to  automatize  the  process  of  creating 
subgroups  of  patients  with  similar  characteristics. 

The  mathematical  principles  that  govern  RST  rely  on  the 
relations  between  objects.  Using  RST,  we  analyze  and 
evaluate  all  possible  relations  between  patients  to  obtain 
the  minimum  and  dispensable  information  required  to 
generate  homogeneous  subgroups  of  patients  (i.e. 
patients  with  similar  characteristics).  We  envision  our 
methodology  to  operate  in  an  automatic  manner  without 
the  researcher  intervention  in  selecting  those 
characteristics  that  matter  in  grouping  patients  for  meta¬ 
analyses. 

II.  Methodology 

A.  Dataset 

Our  dataset  consists  of  individual  patient  data  collected 
from  nine  randomized  trials  studying  the  effect  of 
Allogeneic  Peripheral  Blood  Stem-cell  transplantation 
(PBSCT)  compared  to  Bone  Marrow  transplantation 
(BMT)  in  the  management  of  hematologic  malignancies 
[16].  In  total,  1,111  patients  were  enrolled.  Records  of 
44  patients  containing  missing  information  were 
removed  leaving  the  dataset  with  1067  complete  cases. 

Table  1  describes  the  details  of  our  dataset. 


B.  Rough  Set  Theory 

In  RST,  a  dataset  is  represented  by  an  information 
system  defined  as  a  pair  S  —  ( [U,A )  where  U  is  a  non¬ 
empty  finite  set  of  objects  that  in  our  case  represents  the 
1,111  patients.  The  set  A  represents  a  non-empty  finite 
set  of  attributes  called  the  condition  attributes  that  corresponds  to  the  characteristics  of  each  patient.  For 
every  attribute  a  £  A,  the  function  U  ->  Va  makes  a  correspondence  between  an  object  (i.e.  a  patient)  in  U  to 
an  attribute  value,  which  is  called  the  value  set  of  a.  For  example,  from  table  1,  the  value  of  the  attribute 


Table  1.  Dataset  description 

Variable 

Description 

Categories 

% 

0:  <20 

6.25  % 

Age 

Patient  age 

1:  (20,40] 

2:  (40,  65] 

47.82% 

45.93% 

Gender 

Patient 

1 :  Male 

59.66% 

gender 

2:  Female 

40.34% 

Acute  lymphoblastic 
leukemia  (ALL) 

Acute  myelogenous 
leukemia  (AML) 

12.5% 

33.52% 

Chronic  lymphocytic 
leukemia  (CLL) 

Chronic  myelogenous 

0.28% 

43.47% 

Diag 

Diagnosis 

category 

leukemia  (CML) 

Hodgkin’s  disease  (HD) 
Idiopathic  myelofibrosis 
(IMF) 

Myelodysplastic 
symdrome  (MDS) 

Multiple  myeloma  (MM) 
Non-hodking  lymphoma 
(NHL) 

0.09% 

0.76% 

5.87% 

1.04% 

2.46% 

0:  Favorable  (early-stage 

74.62% 

StatTrans 

Diagnosis 

status 

disease) 

1 :  Unfavorable  (late-stage 
disease) 

25.38% 

Methotrexate 

1:  Yes 

43.84% 

Mtx 

for  GVHD 
prophylaxis 

0:  No 

56.15% 

CondReg 

Conditioning 
regimen  used 

1 :  Total  body  irradiation 
based  (TBI) 

2:  Non  TBI  based 

41.19% 

58.81% 

Use  of  post- 

1:  G-CSF 

58.14% 

GrowthFac 

transplantati 
on  growth 
factor 

0:  not  used 

41.85% 

Alloc 

Treatment 

1:  PBSCT 

2:  BMT 

49.05% 

50.95% 

BR 

5.30% 

US1 

16.29% 

No 

5.78% 

Trial 

Origin  of  the 
study 

SA 

FR 

5.10% 

9.56% 

EBMT 

30.21% 

CAN 

20.36% 

US2 

1.70% 

UK 

3.69% 

Death 

Overall 

0:  Survive 

59.75% 

survival 

1 :  Death 

40.25% 
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‘Age”  can  be  0,  1  or  2  for  a  given  patient.  A  dataset  including  an  outcome  variable  d  £  A,  is  termed  as 


decTsmn^ysTem^IeKiucHS^^^^T^^^c^^^l^IecTslmmiTTnEuf^fHru^IuftH^IT^^maET^^neaTTy 

representing  the  overall  survival  of  a  patient  given  the  characteristics  described  in  A. 


C.  Indiscernibility  and  discernibility  relations 

Two  objects  (e.g.  patients)  it,  it'  6  U  are  indiscernible  with  respect  to  a  set  of  condition  attributes  B  Q  A  if 
they  have  exactly  the  same  values  in  all  attributes,  i.e:  a(it)  =  a(it')  V  a  G  B.  This  relation  is  called 
indiscernibility  relation  and  is  defined  as: 

IND(B)  =  {(u,u')  E  U2:Va  E  B,a(u)  =  a(u')}  V  B  Q  A  (1) 


The  indiscernibility  relation  captures  the  redundant  information  in  the  dataset.  Every  subset  B  Q  A,  can  be 
used  for  constructing  this  relation,  however,  only  subsets  that  maintain  the  structure  of  the  original  dataset, 
i.e:  IND(B )  =  IND(A),  are  considered  appropriate.  Such  a  subset  B  q  a,  is  termed  as  an  exact  reduct.  In  the 
case  that  it  would  not  be  possible  to  obtain  an  exact  reduct,  approximated  reducts  with  acceptable  quality  of 
approximation  are  considered.  The  quality  of  approximation  (aB)  of  a  reduct  B  quantifies  the  proportion  of 
objects  correctly  allocated  in  a  decision  class  by  using  only  the  attributes  in  B,  i.e: 


_  |POS(B)| 

"  ltd 


(2) 


where,  POS(B )  is  the  set  of  all  objects  correctly  assigned  to  the  right  decision  class.  In  general,  the  higher 
the  value  of  aB,  the  more  desirable  the  reduct  is  for  constructing  homogeneous  subgroups. 


On  the  other  hand,  the  discernibility  relation  accounts  for  differences  between  objects  in  terms  of  their 
attribute  values,  i.e: 

DISDS(B)={(u,u')EU2-.3aEB,a{u)±  a{u')}  VBEA  (3) 


III.  Identifying  homogeneous  subgroups  in  individual  patient  dataset 


We  use  the  indiscernibility  relation  to  build  homogenous  subgroups  based  on  patients  with  the  same 
characteristics  and  we  use  the  discernibility  relation  to  explore  the  characteristics  that  differentiate  each 
subgroup.  Fig.  1  depicts  an  overview  of  the  proposed  methodology,  which  is  comprised  of  4  processes:  1. 
Obtain  reducts;  2.  Create  homogeneous  groups;  3.  Regroup  based  on  similarities;  and  4.  Evaluate  groups’ 
heterogeneity. 
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Figure  1.  Overview  of  the  RST 


methodology  for  identifying  homogeneous  subgroups  in  IPD 


Obtaining  reducts:  First,  we  use  the  indiscernibility  relation  IND(B )  to  obtain  an  appropriate  subset  of 
condition  attributes  B  as  the  basis  to  generate  the  homogeneous  subgroups  of  patients.  To  find  this 
subset  of  attributes  (reducts),  we  use  approximated  solutions  described  in  [17].  In  our  dataset,  the  set 
B  —  {Age,  Diag,  StatTrans}  stands  as  the  approximated  reduct  with  the  highest  quality  of 
approximation  (aB  =  0.  71)  among  all  the  generated  reducts. 

Homogeneous  groups:  The  indiscernibility  relation  partitions  the  IPD  in  32  disjoint  homogeneous 
subgroups  with  around  40%  of  them  containing  less  than  10  patients.  Subgroups  with  small  number 
of  patients  do  not  include  patients  from  all  trials  and  are  unsuitable  for  an  individual  patient  meta¬ 
analysis. 

Regrouping  process:  We  obtain  subgroups  with  a  larger  number  of  patients  by  merging  smaller 
subgroups  based  on  a  similarity  relationship.  The  similarity  relation  [18]  is  defined  as  a  less  rigorous 
version  of  the  indiscernibility  relation  and  is  subject  to  a  threshold  value  that  allows  small  differences 
considered  insignificant.  Formally,  we  define  the  similarity  relation  between  subgroups  as: 


#i  SIM By  g2  iff  >  y ,  V  glt  g2  E  U/IND(B )  and  u  G  gxand  u'  G  g2 

(4) 


Where,  X  —  {a  E  B:  a(it)  =  a(it')}  and  y  e  [0,1]  is  the  similarity  threshold. 

Since  comparing  all  possible  combinations  between  two  groups  to  detennine  their  similarity  is  a  complex 
process  we  use  a  more  straightforward  procedure  consisting  in  evaluating  the  differences  between 
subgroups.  Then,  subgroups  having  similar  differences  to  the  rest  of  the  subgroups  are  combined  resulting  in 
one  homogenous  group. 


We  define  a  discernibility  matrix  of  subgroups  MB,  where  each  cell  MB(gL,  gj)  represents  the  number  of 
attributes  in  B,  whose  values  distinguish  subgroup  gt  from  subgroup  gj,  i.e: 

MB(gi,gj)  —  {\Dif\},  where  Dif  =  {  a  G  B:  a(u )  A  a(it')}  V  gv  g2  E  U/IND(B )  and  u  G  g^nd  u'  G 

92  (5) 

Fig.  2  shows  a  portion  of  the  discernibility  matrix  obtained  for  the  32  homogenous  subgroups. 


IV.  results 

The  initial  32  homogeneous  subgroups  are  regrouped  based  on  similarities  in  the  number  of  attributes  that 
distinguish  them  from  the  rest  of  groups.  We  chose  ay  —  0.8  value  (Equation  5)  as  a  threshold  parameter  of 
similarity  to  minimize  the  number  of  homogeneous  groups  by  allowing  some  degree  of  differences.  For 
example,  the  initial  subgroups  18,  19  and  20  (Fig.  2)  can  be  regrouped  since  there  are  no  more  than  20%  of 
differences  across  their  corresponding  rows.  In  other  words,  the  three  subgroups  have  similar  distances,  in 
terms  of  differences,  to  the  rest  of  groups.  As  a  result,  the  32  homogeneous  groups  are  gathered  in  three 
groups.  Table  2  shows  the  homogenous  groups  resultant  after  the  regrouping  process.  The  mean  number  of 
patients  in  each  group  is  equal  to  355  with  a  standard  deviation  of  39. 15. 
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Figure  2.  A  portion  of  the  discemibility  matrix  obtained  for  the  homogeneous  groups.  Each  cell  shows  the  number  of  attributes  that  differentiate  between  each  pair 
of  subgroups. 


Table  2.  Three  homogeneous  groups  obtained  from  the  regrouping 
process 

Group 

number 

Original  group 

Number  of  patients 

1 

10.  12,  14,  17 

392 

2 

21,23,26,  29 

359 

3 

1-9,1 1,13,15-16,18- 
20,22,24-25,27-28,30-32 

314 

The  obtained  homogeneous  groups  (Table  2)  contain  similar  distributions  in  terms  of  trials,  diagnosis  and 
treatment.  The  statistical  heterogeneity  (I2)  indicate  a  negligible  heterogeneity  value  for  all  the  three  groups 
(16.8%  in  group  1,  0%  for  group  2,  and  0%  for  group  3),  which  suggests  that  all  groups  are  indeed 
homogeneous. 


V.  Conclusions 

In  this  preliminary  work,  we  utilized  a  methodology  typically  found  in  engineering  applications  to  solve  a  problem  that  exists  in 
the  realm  of  evidence-based  medicine.  Researchers  who  perform  evidence  synthesis  are  faced  with  the  challenge  of  detecting 
heterogeneity  between  clinical  trials  and  then  explaining  it  by  hypothesizing  standards  of  similarity.  However,  there  is  no 
commonly  accepted  approach  to  identify  similarities  between  trials  and  meta-analysts  resolve  to  ad-hoc  solutions.  Here  we 
presented  a  methodology  based  on  Rough  Set  Theory  that  has  the  potential  to  automatize  and  standardize  this  process. 

We  demonstrated  the  effectiveness  of  our  methodology  using  a  sample  dataset  containing  1,111  patients  from  9  different 
trials.  We  showed  that  were  able  to  identify  the  appropriate  patient  characteristics  to  construct  homogenous  groups  that  presented 
similar  proportion  of  trials,  controls  (diagnosis)  and  interventions  (treatments)  in  accordance  to  the  fundamental  doctrine  of  meta¬ 
analysis.  Thus,  these  groups  are  suitable  to  derive  the  pooled  estimate  of  treatment  effects  in  individual  patient  meta-analysis. 

Other  applications  of  this  methodology  include  identifying  subgroups  of  patients  that  need  different  treatments,  patients  with 
differential  responses  to  therapy,  or  patients  that  belong  to  different  risk  groups.  Analyzing  the  effect  of  treatment  in  each 
subgroup  is  very  important  for  personalized  healthcare.  Our  intention  is  to  compare  this  methodology  with  similar  approaches  in 
other  data  sets. 

Finally,  this  is  a  preliminary  work  and  presents  limitations.  Particularly,  we  have  not  investigated  the 
effects  of  our  methodology  in  the  results  of  meta-analysis,  which  we  intent  to  do  in  the  future.  Other  future 
research  includes  generalization  of  our  methodology  to  accommodate  clinical  trial  data  in  addition  to 
individual  patient  data. 
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Purpose :  The  acceptable  regret  model  postulates  that  under  specific  circumstances  decision  makers  may 
tolerate  wrong  decisions  ( Med  Hypotheses,  53,  253-9;  PLoS  Med,  4,  e26;  Med  Dec  Making,  28,  540-553; 
Med  Dec  Making,  29,  320-322).  The  purpose  of  this  work  is  to  empirically  evaluate  the  acceptable  regret 
model  of  decision-making  in  end-of-life  care  settings,  where  terminally  ill  patients  consider  seeking  curative 
treatment  or  accepting  hospice/palliative  care. 


Methods ;  We  conducted  interviews  with  24  patients  enrolled  in  the  study  assessing  their  preferences  about 
end-of-life  treatment  choices.  After  providing  information  about  their  life  expectancy  and  assessing  the 
overall  regret  of  potentially  wrong  choices  ( BMC  Med  Inform  Decis  Mak,  10,  51),  we  elicited  the  patients’ 
level  of  acceptable  regret.  We  first  assessed  the  patients’  tolerance  for  wrongly  accepting  hospice  care  and 
then  measured  the  patients’  tolerance  toward  continuing  unnecessary  treatment.  For  the  purposes  of  our 
study,  a  treatment  was  considered  unnecessary  if  a  patient  dies  within  6  months  of  the  treatment.  Accepting 
hospice  care  was  considered  a  wrong  decision  if  a  patient  survives  longer  than  6  months  after  the  referral  to 
hospice.  We  elicited  acceptable  regret  levels  to  compute;  1)  the  probability  of  death  above  which  a  patient 
would  tolerate  wrongly  accepting  hospice  care  and  2)  the  probability  of  death  below  which  the  patient  would 
tolerate  unnecessary  treatment  ( BMC  Med  Inform  Decis  Mak,  10,  51;  Med  Dec  Making,  28,  540-553). 

Results'.  We  found  that  the  median  probability  of  death  above  which  a  decision  maker  would  tolerate 
wrongly  accepting  hospice  care  is  98%,  while  the  median  probability  of  death  below  which  a  decision  maker 
would  tolerate  unnecessary  treatment  is  4%.  We  also  found  that  the  levels  of  acceptable  regret  measured  for 
wrong  hospice  referral  (mean=1.68;  SD=2.3;  min=0;  max=7.28)  are  similar  to  the  levels  of  acceptable  regret 
measured  for  unnecessary  treatment  (mean=1.27;  SD=1.97;  min=0;  max=6.58)  (KW  test;  p=0.73)  indicating 
that  acceptable  regret  levels  for  either  of  wrong  decisions  is  felt  similarly.  Our  results  are  independent  of  the 
estimated  probability  of  death  communicated  to  patients  prior  to  the  acceptable  regret  interview. 


Conclusions:  We  have  elicited  preliminary  empirical  data  that  corroborated  the  acceptable  regret  theory.  Our 
results  may  explain  why  has  been  so  difficult  to  provide  palliative  care  in  the  end  of  life  setting. 


Downloaded  from  bmjopen.bmj.com  on  October  20,  2014  -  Published  by  group.bmj.com 


Open  Access  Research 


BMJ  Open  From  hospice  to  hospital:  short-term 
follow-up  study  of  hospice  patient 
outcomes  in  a  US  acute  care  hospital 
surveillance  system 


Elizabeth  Barnett  Pathak,1  Sarah  Wieten, 1,2  Benjamin  Djulbegovic1 


To  cite:  Pathak  EB,  Wieten  S,  Dr  Elizabeth  Barnett  Pathak; 

Djulbegovic  B.  From  hospice  epathak@health.usf.edu 

to  hospital:  short-term 

follow-up  study  of  hospice 

patient  outcomes  in  a  US 

acute  care  hospital 

surveillance  system.  BMJ 

Open  2014;4:e005196. 

doi:1 0.1 1 36/bmjopen-2014- 

005196 


*  Prepublication  history  for 

this  paper  is  available  online. 
To  view  these  fles  please 
visit  the  journal  online 
(http://dx.doi.Org/10.1 136/ 
bmjopen-201 4-0051 96). 


Received  5  March  2014 
Revised  1  July  2014 
Accepted  4  July  2014 


CrossMark 


'Division  of  Evidence-Based 
Medicine,  Morsani  College  of 
Medicine,  University  of  South 
Florida,  Tampa,  Florida,  USA 
department  of  Philosophy, 
University  of  South  Florida, 
Tampa,  Florida,  USA 


ABSTRACT 

Objectives:  In  the  USA,  there  is  little  systematic 
evidence  about  the  real-world  trajectories  of  patient 
medical  care  after  hospice  enrolment.  The  objective  of 
this  study  was  to  analyse  predictors  of  the  length  of  stay 
for  hospice  patients  who  were  admitted  to  hospital  in  a 
retrospective  analysis  of  the  mandatorily  reported 
hospital  discharge  data. 

Setting:  All  acute-care  hospitals  in  Florida  during  1 
January  2010  to  30  June  2012. 

Participants:  All  patients  with  source  of  admission 
coded  as  'hospice'  (n=2674). 

Primary  outcome  measures:  The  length  of  stay  and 
discharge  status:  (1 )  died  in  hospital;  (2)  discharged 
back  to  hospice;  (3)  discharged  to  another  healthcare 
facility;  and  (4)  discharged  home. 

Results:  Patients  were  elderly  (median  age=81)  with  a 
high  burden  of  disease.  Almost  half  died  (46%),  while 
the  majority  of  survivors  were  discharged  to  hospice 
(80%  of  survivors,  44%  of  total).  A  minority  went  to  a 
healthcare  facility  (5.6%)  or  to  home  (5.2%).  Only 
9.2%  received  any  procedure.  Respiratory  services 
were  received  by  29.4%  and  16.8%  were  admitted  to 
the  intensive  care  unit.  The  median  length  of  stay  was 
1  day  for  those  who  died.  In  an  adjusted  survival 
model,  discharge  to  a  healthcare  facility  resulted  in  a 
74%  longer  hospital  stay  compared  with  discharge  to 
hospice  (event  time  ratio  (ETR)=1 .74,  95%  Cl  1.54  to 
1.97  p<0.0001),  with61%  longer  hospital  stays  among 
patients  discharged  home  (ETR=1 .61 , 95%  Cl  1.39  to 
1.86  p<0.0001).  Total  financial  charges  forall  patients 
exceeded  $25  million;  10%  of  patients  who  appeared 
to  exit  hospice  incurred  32%  of  the  charges. 
Conclusions:  Our  results  raise  significant  questions 
about  the  ethics  and  pragmatics  of  end-of-life  medical 
care,  and  the  intentions  and  scope  of  hospices  in  the 
USA.  Future  studies  should  incorporate  prospective 
linkage  of  subjective  patient-centred  data  and  objective 
healthcare  encounter  data. 
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Hospice  care  is  a 
specialised  approach 
to  end-of-life  medical 
care  that  emphasises 


■  One  of  the  largest  studies  until  now  to 

explore  acute-care  hospitalisations  of  hospice 
patients,  a  neglected  aspect  of  the 
dying  patient’s  experience. 

■  Recent,  unbiased  and  comprehensive  surveil¬ 
lance  data  from  the  fourth  largest  state  in 
the 

USA  (Florida)  were 
analysed. 

■  Detailed  analyses  of  morbidities,  in-hospital  pro¬ 
cedures,  length  of  hospital  stay  and 
financial 

charges  were  included  for  four  distinct 
patient  groups:  patients  who  died,  those  who 
returned  to  hospice,  those  who  were 
transferred  and  those  who  went  home  without 
hospice  care. 

■  Limitations  included  probable  under-reporting  of 
hospice  as  source  of  admission;  inability  to 
dis¬ 
tinguish  which  morbidity  was  the  terminal 
illness  and  lack  of  information  about 
patient/caregiver  preferences  for  end-of-life 
care  and  place  of  death. 


quality  of  life,  pain  management  and 
symptom  alleviation  for  terminally  ill 
patients.  In  the  USA,  hospice  care  is 
usually  provided  in  the  patient’s  home  or  in  a 
non-  hospital  facility,  and  it  typically 
excludes  curative-intent  medical  or  surgical 
interven-  tions.  The  goal  of  hospice  care 
is  not  to  prolong  life  or  postpone  dying, 
but  rather  to  achieve  a  ‘good  death.’ 
Generally,  the  pre-  ferred  length  of 
enrolment  in  hospice  is  about  6  months 
before  death,  but  some  guidelines  point 
to  a  recommended  stay  of 
1  year.1  As  a  stipulation  for  receiving 
hospice 

benefits  from  most  US  insurance  payers, 
the  patient  relinquishes  access  to  curative 
inter-  ventions,  although  the  patient  can  opt 
out  of  hospice  at  any  time  to  regain  this 
access.  The  idealised  patient  trajectory  that 
is  evoked  by  hospice  advocates  is  that  after 
patients  and  families  come  to  terms  with 
the  terminal  nature  of  the  illness,  patients 
will  live  out  the  remainder  of  their  days 
peacefully  at  home 
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or  in  a  hospice  facility  with  minimal  medical  interven¬ 
tion  prior  to  death. 

However,  there  is  very  little  systematic  evidence  about 
the  real-world  trajectories  of  patient  medical  care  after 
hospice  enrolment  in  the  USA.  Instead,  previous  hospice 
studies  have  focused  on  whether  the  use  of  hospice  or 
advanced  directives  decrease  the  cost  of  end-of-life  care"-4 
or  increase  the  quality  of  life  before  death.5  6  These 
studies  of  cost  and  quality  of  life  have  generally  assumed 
the  idealised  trajectory  of  hospice  care,  without  consider¬ 
ing  the  possible  impact  of  hospitalisations.  Previous 
reports  on  the  phenomenon  of  post-hospice  hospital 
admissions  have  been  small  cohorts7  or  single-centre 
studies’’  and  have  found  that  hospitalisation  of  hospice 
patients  can  not  only  be  costly  and  largely  preventable  but 
also  that  positive  patient  outcomes  for  interventions  on 
non-terminal  conditions  (like  hip  fractures)  might  explain 
some  hospice  patient  hospital  admissions. 

In  this  study,  one  of  the  largest  so  far  of 
hospice-to-hospital  patients,  we  analysed  very  recent  data 
from  a  statewide  hospital  surveillance  system  for  all  inpa¬ 
tients  whose  source  of  admission  was  coded  as  ‘hospice.’ 
We  report  patient  demographics  and  clinical  character¬ 
istics;  hospitalisation  procedures,  duration  and  out¬ 
comes;  and  detailed  financial  charges.  Finally,  we 
modelled  predictors  of  length  of  hospital  stay  for  the 
majority  of  these  terminally  ill  hospice  patients  who  sur¬ 
vived  to  discharge.  Given  that  one  of  the  core  goals  of 
hospice  programmes  in  the  USA  is  to  avoid  unnecessary 
and  futile  medical  care  at  the  end  of  life,  we  presumed 
that  a  long  hospital  stay  would  constitute  a  negative  and 
unwanted  outcome  for  most  hospice  patients. 

METHODS 

Our  study  population  consisted  of  all  patients  at  Florida 
acute  care  hospitals  whose  source  of  admission  was 
coded  ‘hospice’  in  the  state  hospital  discharge  surveil¬ 
lance  system.  Other  common  sources  of  admission 
include  emergency  department,  hospital  transfer, 
patient  home  and  skilled  nursing  facility.  In  2010,  the 
reporting  requirements  were  modified  and  a  new  code 
to  identify  hospice  patients  was  added.  In  this  study,  we 
aggregated  2.5  years  of  data  (1  January  2010  to  30  June 
2012)  for  analysis. 

Hospitals  in  Florida  are  mandated  by  state  law  to 
submit  detailed  discharge  records  of  all  patients. 
Consequently,  these  data  can  be  considered  a  surveil¬ 
lance  system  with  100%  coverage.  Data  items  available 
include  patient  demographics,  payer,  length  of  stay, 
admission  diagnosis,  principal  diagnosis,  up  to  30  sec¬ 
ondary  diagnoses,  principal  and  secondary  procedure 
codes  and  detailed  financial  charge  data. 

We  analysed  data  for  four  distinct  groups  based  on  dis¬ 
charge  status/destination.  The  groups  were  (1)  patients 
who  died  prior  to  discharge;  (2)  patients  who  were  dis¬ 
charged  back  to  hospice  (either  home  hospice  care  or  a 
hospice  facility);  (3)  patients  who  were  discharged  to  a 


healthcare  facility  (including  transfer  to  another  acute- 
care  hospital,  skilled  nursing  facility,  intermediate-care, 
long-term  care  or  rehabilitation  facility);  and  (4)  patients 
who  were  discharged  home  (either  with  or  without  home 
health  assistance,  but  without  hospice).  For  each  of  our 
four  study  groups,  we  calculated  prevalence  rates  for 
common  admission  diagnoses,  any-mention  diagnoses 
and  procedures.  Specifically,  ‘any-mention  diagnosis’  was 
based  on  any  mention  of  a  condition  in  any  of  32  diagno¬ 
sis  fields  (including  admission  diagnosis,  principal  diag¬ 
nosis  and  secondary  diagnosis  fields).  Multiple  ICD-9-CM 
codes  were  combined  into  categories  in  some  cases. 
Patients  with  a  diagnosis  of  heart  failure  included  those 
with  chronic,  acute  exacerbation  or  both.  A  diagnosis  of 
psychosis  included  those  who  were  suffering  from  serious 
behavioural  effects  of  Alzheimer’s  disease.  While  the 
principal  diagnosis  code  is  intended  to  capture  the  most 
serious  medical  condition  suffered  by  the  patient  (eg, 
lung  cancer),  the  admission  diagnosis  code  is  intended 
to  identify  why  the  patient  was  admitted  as  an  inpatient 
to  the  hospital  (eg,  acute  respiratory  failure). 
Consequently,  admission  diagnosis  codes  may  include  dis¬ 
eases,  symptoms,  signs  or  ‘V-codes’,  which  are  ICD-9-CM 
codes  that  describe  social  circumstances,  medical  history 
or  other  pertinent  patient  information  not  captured  by 
traditional  organ  system-based  disease  codes.  For 
example,  the  code  V66.7  indicates  an  episode  of  palliative 
healthcare. 

Payer  categories  included  Medicare  (federal  govern¬ 
ment  health  insurance  for  the  elderly),  Medicaid 
( federal/state  health  insurance  for  low-income  persons), 
Tricare  (federal  health  insurance  for  military  personnel 
and  their  dependants),  commercial  (all  non-government 
health  insurance  plans  including  employer-provided 
plans)  and  other  (including  self-pay/underinsured, 
charity,  worker’s  compensation  and  other  miscellaneous 
small  programmes). 

Procedures  (up  to  31  per  patient)  were  identified  by 
ICD-9-CM  procedure  codes.  Procedures  could  include 
major  therapeutic  interventions  (eg,  surgery),  minor 
therapeutic  interventions  (eg,  breathing  treatment), 
diagnostic  procedures  (eg,  MRI,  CT  scan  and  colonos¬ 
copy)  and  minor  routine  procedures  (eg,  insertion  of 
catheters). 

Given  that  hospital  financial  coverage  is  constrained 
by  some  payers  for  hospice  patients  based  on  length  of 
stay,  we  examined  predictors  of  duration  of  hospital  stay 
using  a  Weibull  accelerated  failure  (event)  time  survival 
model. The  Weibull  model  permits  calculation  of 
‘event  time  ratios  (ETRs)’,  which  are  a  more  appropriate 
measure  of  effect  for  this  study  than  HRs.  This  is 
because  our  primary  interest  is  not  in  whether  or  not  an 
event  occurs  (all  patients  eventually  leave  the  hospital), 
but  in  whether  there  are  factors  that  accelerate  or  delay 
time  to  event  (ie,  days  until  discharge).  ETRs  provide  a 
proportional  measure  of  direct  impact  on  time  to  event 
(eg,  an  ETR  of  1.25=a  25%  increase  in  the  length  of  hos¬ 
pital  stay).  We  first  fit  a  model  with  a  large  number  of 
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potential  predictors.  For  the  final  model,  we  retained  all 
covariates  with  an  initial  p  value  <0.15. 

In  addition,  some  payers  will  cover  occasional  hospita¬ 
lisations  of  limited  duration  (eg,  S5  days)  for  hospice 
patients,  for  the  purposes  of  respite  care  or  medical 
attention  to  a  condition  which  is  not  the  terminal 
illness.1  11  Therefore,  we  used  a  multivariate  logistic 
regression  model  to  examine  the  dichotomous  outcome 
of  long  (£6  days)  vs  short  (£5  days)  hospital  stays 
among  survivors. 

Finally,  we  examined  detailed  financial  charge  data 
available  for  each  patient  as  a  proxy  for  costs.  We  evalu¬ 
ated  which  charge  centres  (eg,  pharmacy  and  intensive 
care)  were  contributing  to  total  costs  both  overall  and 
stratified  by  patient  discharge  status.  We  report  the  per¬ 
centage  of  patients  with  a  non-$0  charge  in  a  given 
charge  centre,  and  the  median  and  99th  centile  dollar 
values  (calculated  based  only  on  the  distribution  of 
non-$0  values). 

This  study  was  classified  as  non-human  subjects 
research  by  the  USF  Institutional  Review  Board  because 
it  relied  solely  on  de-identified  secondary  database 
analyses. 

RESULTS 

Patient  discharge  status 

There  were  a  total  of  2764  patients  whose  source  of 
admission  was  reported  as  ‘hospice’  during  the  period  1 
January  2010  to  30  June  2012  at  acute -care  general  hos¬ 
pitals  in  Florida.  The  discharge  status  of  these  patients  is 
shown  in  figure  1.  Almost  half  of  these  patients  died 
prior  to  discharge  (46%),  while  the  majority  of  survivors 
were  discharged  back  into  a  hospice  programme  (80% 
of  survivors,  44%  of  total).  Patients  discharged  home 
(5.2%)  included  those  who  were  scheduled  to  receive 
home  healthcare. 

Patient  characteristics,  diagnoses  and  length  of  stay 
Characteristics  of  patients  by  discharge  status  are  shown 
in  table  1.  The  overall  median  age  was  81  years  (range 
0—106  years);  however,  non-trivial  minorities  of  each 


group  were  aged  <65  years  (12.8-45.7%),  and  5%  of  the 
patients  discharged  home  were  markedly  younger  than 
the  other  three  groups  (median  age=67).  The  majority 

of  patients  were  White  non-FIispanic  (70.8%),  followed 
by  Flispanic  (19.9%).  The  majority  of  patients  had 
Tricare  health  insurance  (51.5%),  followed  by  other 
(25.4%),  private/commercial  insurance  (16.8%)  and 
Medicare  or  Medicaid  (6.3%). 

As  expected,  these  terminally  ill  patients  suffered  a 
high  burden  of  serious  illness.  The  most  prevalent  diag¬ 
noses  (based  on  any  mention)  are  listed  in  table  1. 
Some  of  these  conditions  were  chronic  diseases  (eg, 

hypertensive  disease,  any  heart  disease,  cancer  and 
chronic  obstructive  pulmonary  disease  (COPD)),  while 
others  probably  reflected  acute  conditions  that  may  have 
precipitated  hospital  admission  (eg,  respiratory  failure, 
acute  renal  failure,  stroke,  pneumonia/influenza  and 
sepsis). 

The  median  length  of  stay  for  all  patients  was  2  days, 
with  a  range  of  0—99  days.  While  only  10.7%  of  hospice 
patients  had  a  stay  of  6  or  more  days,  a  longer  length  of 
stay  was  much  more  prevalent  among  patients  dis¬ 

charged  to  a  healthcare  facility  (32.9%)  or  home 
(21.7%).  In  contrast,  patients  who  died  were  most  likely 
to  have  a  length  of  stay  of  zero  days  (22.8%),  meaning 
that  those  patients  died  on  the  same  day  that  they  were 
admitted  to  the  hospital. 

Admission  diagnoses 

The  top  admission  diagnoses  for  patients  in  our  study 

population  are  depicted  in  figure  2.  For  a  terminally  ill 
patient  suffering  from  multiple  chronic  and  acute  condi¬ 
tions,  the  choice  of  a  single  admission  diagnosis  may  be 
somewhat  arbitrary.  In  aggregate,  however,  these  codes 
provide  a  window  of  insight  into  the  diverse  challenges 
present  in  medical  management  of  the  dying  patient. 
Top  admission  diagnoses  among  patients  who  were  dis¬ 
charged  to  a  healthcare  facility  included  palliative  care 
(16.1%),  cancer  (6%),  psychosis  (4.7%),  altered  mental 
status  (4%)  and  pneumonia/influenza  (3.4%).  Among 
patients  who  were  discharged  home,  top  diagnoses 
included  cancer  (8.7%),  respiratory  symptoms  (7.3%), 
drug/alcohol  dependence  (6.5%)  and  psychosis  (6.5%). 


i  Died  Before  Discharge 
i  Discharged  to  Hospice 
i  Discharged  to  Healthcare  Facility 
i  Discharged  Home 


Percent 


Figure  1  Discharge  outcomes  for  patients  admitted  to  a 
hospital  from  hospice,  Florida  2010—2012  (n=2674). 


Medical  and  surgical  procedures 

Overall,  the  use  of  procedures  was  very  limited  in  this 
population  of  terminally  ill  hospice  patients,  with  only 
9.2%  overall  receiving  any  procedure  (table  2).  Among 
patients  who  died,  95.2%  received  no  procedure.  Invasive 
mechanical  ventilation  was  provided  in  1%  of  these 
patients,  and  non-invasive  mechanical  ventilation  in  0.8%. 
At  least  one  procedure  was  received  by  6.8%  of  patients 
who  were  discharged  to  hospice.  In  this  group,  the  two 
most  common  principal  procedures  were  paracentesis 
(1%)  and  transfusion  of  packed  cells  (0.9%). 

Procedure  use  was  more  common  among  patients  who 
did  not  return  to  hospice  (33.6%  for  those  who  were  dis¬ 
charged  to  a  healthcare  facility  and  41.3%  for  those  who 
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Table  1  Characteristics*  of  patients  admitted  from  hospice  by  final  discharge  status,  Florida  2010-2012 
Died  before  Discharged  to  a 


discharge 

(n=1217) 

Discharged  to 
hospice  (n=1170) 

healthcare  facility 
(n=149) 

Discharged 
home  (n=138) 

Total 

(n=2764) 

Median  age 

80 

81 

84 

67 

81 

Age,  years  (%) 

<64 

16.1 

15.2 

12.8 

45.7 

17.1 

65-74 

18.1 

14.9 

16.1 

18.8 

16.6 

75-84 

29.5 

30.3 

26.2 

21.7 

29.2 

85-94 

31.1 

33.7 

38.3 

9.4 

31.5 

95+ 

5.3 

6.0 

6.7 

4.4 

5.6 

Gender  (%) 

Male 

49.1 

47.5 

38.9 

40.6 

47.4 

Female 

50.9 

52.5 

61.1 

59.4 

52.6 

Race/ethnicity  (%) 

White 

73.7 

69.6 

72.5 

52.9 

70.8 

Hispanic 

17.0 

22.5 

16.8 

26.8 

19.9 

Black 

6.9 

6.7 

9.4 

15.2 

7.4 

Other 

2.4 

1.3 

1.3 

5.1 

2.0 

Payer  (%) 

Medicare/Medicaid 

1.3 

4.2 

35.6 

37.0 

6.3 

Private  insurance 

14.7 

19.2 

6.0 

26.1 

16.8 

Tricare/F  ederal 

57.7 

49.8 

40.3 

23.2 

51.5 

Other 

26.3 

26.8 

18.1 

13.8 

25.4 

12  Most  prevalent  diagnoses"! 
Hypertensive  disease  37.6 

46.4 

62.4 

50.7 

43.5 

Non-ischaemic  heart 

31.1 

32.1 

38.3 

18.8 

31.3 

disease 

Cancer 

28.1 

36.6 

23.5 

19.6 

31.1 

Respiratory  failure 

37.9 

18.8 

17.5 

10.1 

27.0 

Ischaemic  heart 

25.1 

25.0 

33.6 

21.0 

25.4 

disease 

Chronic  obstructive 

23.9 

25.9 

27.5 

21.0 

24.8 

pulmonary  disease 

Heart  failure 

25.4 

22.3 

21.5 

19.6 

23.5 

Psychosis 

15.9 

28.4 

43.6 

28.4 

23.5 

Acute  renal  failure 

16.7 

13.4 

18.1 

10.9 

15.0 

Stroke 

15.6 

11.8 

13.4 

8.7 

13.5 

Pneumonia  or 

14.1 

14.2 

12.1 

2.2 

13.4 

influenza 

Sepsis 

13.6 

8.8 

9.4 

3.6 

10.8 

Median  length  of  stay 

1 

2 

3 

3 

2 

(days) 

Per  cent  with  stay 

9.7 

7.6 

32.9 

21.7 

10.7 

>5  days 

*A11  per  cents  are  column  per  cents. 

"(Based  on  a  mention  in  any  of  the  32  diagnosis  fields.  Each  patient  could  have  multiple  diagnoses. 


were  discharged  home).  Top  procedures  for  hospice 
patients  discharged  to  a  healthcare  facility  were  percutan¬ 
eous  endoscopic  gastrostomy  (PEG)  (3.4%),  venous  cath¬ 
eterisation  (2%)  and  transfusion  of  packed  cells  (2%).  For 
patients  discharged  home,  drug  detoxification  was  the  top 
procedure  (5.1%),  followed  by  PEG  (2.2%). 

Predictors  of  the  hospital  length  of  stay 
We  found  several  significant  predictors  of  length  of  hos¬ 
pital  stay  in  an  accelerated  event  time  survival  analysis  of 
all  hospice  patients  who  survived  to  hospital  discharge 


(n=1457),  as  shown  in  table  3.  ETRs  significantly  <1.00 
indicate  predictors  that  shortened  the  length  of  stay  (ie, 
accelerated  time  to  discharge),  while  ETRs  significantly 
>1.00  indicate  factors  that  delayed  hospital  discharge. 
We  first  fit  a  model  with  a  large  number  of  potential  pre¬ 
dictors.  For  the  final  model,  we  retained  all  covariates 
with  an  initial  p  value  SO.  15.  Potential  predictors  which 
were  NOT  included  in  the  final  model  were:  sex,  COPD, 
ischaemic  heart  disease,  HIV,  hypertensive  disease,  any 
fracture,  respiratory  failure,  drug/alcohol  dependence, 
sepsis,  acute  renal  failure,  end-stage  renal  disease  and 
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A  Palliative  care 
Cancer 
Respiratory  failure 
Stroke 
Heart  failure 
Sepsis 

Respiratory  symptoms 
COPD 

Pneumonia/influenza 
Aspiration  pneumonia 
Acute  renal  failure 
Anoxic  brain  damage 


Percent 


Palliative  care 
Cancer 
Psychosis 
Altered  mental  status 
Pneumonia/influenza 
Respiratory  symptoms 
Stroke 

Adult  failure  to  thrive 
COPD 
Bone  fracture 
Respiratory  failure 
Anemia 
Aspiration  pneumonia 
Gl  hemorrhage 
Heart  failure 


B  Cancer 

Palliative  care 
Stroke 
Heart  failure 
Respiratory  failure 
Respiratory  symptoms 
Sepsis 
Psychosis 
Pneumonia/influenza 
COPD 
Senility 
Acute  renal  failure 


D  Cancer 

Respiratory  symptoms 
Drug  dependence 
Psychosis 
Palliative  care 
Stroke 
Heart  failure 
Ischemic  heart  disease 
Dehydration 
Diverticulitus 
Rehabilitation  care 
COPD 
Mental  disorder 


Percent 


Percent 


Figure  2  (A)  Top  admission  diagnoses  for  hospice  patients  who  died  prior  to  hospital  discharge  (n=1217).  (B)  Top  admission 

diagnoses  for  hospice  patients  who  were  discharged  back  to  hospice  (n=1170).  (C)  Top  admission  diagnoses  for  hospice 
patients  who  were  discharged  to  a  healthcare  facility  (n=149).  (D)  Top  admission  diagnoses  for  hospice  patients  who  were 
discharged  home  without  hospice  care  (n=138).  COPD,  chronic  obstructive  pulmonary  disease;  GI,  gastrointestinal. 


senility.  The  only  patient  characteristic  that  shortened 
hospital  stay  was  older  age.  For  example,  compared  with 
those  S64  years  of  age,  patients  aged  85—94  years  had 
hospital  stays  that  were  35%  shorter  (ETR  0.65,  95%  Cl 
0.58  to  0.74,  p<0.0001).  In  contrast,  several  factors 
lengthened  hospital  stays,  including  Medicare/Medicaid 
(74%  longer  stays  than  those  with  private  insurance), 
discharge  home  (61%  longer  stays  than  those  who 


returned  to  hospice)  or  to  another  healthcare  facility 
(74%  longer  stays  than  those  who  returned  to  hospice) 
and  Flispanic  ethnicity  (12%  longer  stays  than 
non-Hispanic  Whites).  Clinical  diagnoses  that  increased 
the  length  of  stay  included  medical  complications/infec¬ 
tions  (43%  longer  stays  than  those  without  these  diagno¬ 
ses),  stroke  (29%  longer  stays),  heart  failure  (18% 
longer  stays)  and  psychosis  (13%  longer  stays).  A 


Table  2  Most  prevalent  principal  procedures* 

for  hospice  patients  admitted  to  a  hospital,  Florida  2010-2012 

Died  befoe 
discharge 

Discharged  to 

Discharged  to  a 
healthcare  facility 

Discharged 

Total 

(n=1217) 

hospice  (n=1170) 

(n=149) 

home  (n=138) 

(n=2764) 

Per  cent  with  no  procedure 

95.2 

93.2 

66.4 

58.7 

91.8 

Mechanical  ventilation  for 

n=12 

n=8 

n=l 

n=l 

n=22 

<96  h 

Venous  catheterisation 

6 

8 

3 

1 

18 

Blood  transfusion 

4 

11 

3 

0 

18 

Non-invasive  ventilation 

10 

5 

2 

0 

17 

Paracentesis 

1 

12 

1 

0 

14 

Enteral  feeding 

9 

2 

0 

0 

11 

Percutaneous  endoscopic 

1 

1 

5 

3 

10 

gastrostomy 

Insertion  of  an  endotracheal 

4 

2 

2 

0 

8 

tube 

Drug  detoxification 

0 

0 

0 

7 

7 

Flaemodialysis 

1 

3 

1 

0 

5 

Radiation  therapy 

0 

3 

1 

1 

5 

Interruption  of  vena  cava 

0 

2 

1 

1 

4 

Mechanical  ventilation  S96  h 

1 

1 

2 

0 

4 

*A11  per  cents  are  column  per  cents. 

Each  patient  had 

one  (or  no)  principal  procedure  listed. 
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Table  3  Predictors  of  the  length  of  hospital  stay  among 
hospice  patients  who  survived  to  discharge  (n=1457): 
multivariate  accelerated  event  time  survival  analysis 
results 

Patient  predictors 

Event  time  ratio  (95%  Cl) 

(p  value) 

Discharge  destination 

Hospice 

1.00  (referent) 

Healthcare  facility 

1.74  (1.54  to  1.97)  (<0.0001 ) 

Home 

1.61  (1.39  to  1.86)  (<0.0001 ) 

Age,  years 

<64 

1.00  (referent) 

65-74 

0.73  (0.64  to  0.84)  (<0.0001 ) 

75-84 

0.69  (0.61  to  0.78)  (<0.0001) 

85-94 

0.65  (0.58  to  0.74)  (<0.0001 ) 

95+ 

0.72  (0.61  to  0.86)  (0.0003) 

Race/ethnicity 

White 

1.00  (referent) 

Hispanic 

1.12  (1.03  to  1.22)  (0.01) 

Black 

1.06  (0.92  to  1.22)  (0.44) 

Other 

1.82  (1.37  to  2.42)  (<0.0001 ) 

Payer 

Private  insurance 

1.00  (referent) 

Medicare/Medicaid 

1.74  (1.50  to  2.02)  (<0.0001 ) 

Tricare/Federal 

1.21  (1.09  to  1.34)  (0.0002) 

Other 

0.93  (0.84  to  1.04)  (0.21) 

Clinical  diagnoses  (present 

vs  absent) 

Medical  complications/ 

1.43  (1.18  to  1.73)  (0.0002) 

infections 

Stroke 

1.29  (1.15  to  1.44)  (0.0001) 

Heart  failure 

1.18  (1.07  to  1.30)  (0.001) 

Psychosis 

1.13  (1.04  to  1.23)  (0.04) 

Pneumonia/influenza 

1.11  (1.00  to  1.24)  (0.06) 

Other  heart  disease* 

0.92  (0.85  to  1.01)  (0.08) 

Cancer 

1.07  (0.98  to  1.17)  (0.11) 

*Excludes  ischaemic  heart  disease  and  heart  failure.  Includes 

endocarditis,  pericarditis,  valve  disease,  cardiomyopathy, 

pulmonary  hypertension  and  other  specified  and  ill-defined 

diseases  of  the  heart. 

diagnosis  of  cancer  did  not  affect  the  length  of  hospital 
stay  (ETR  1.07,  p=0.11). 

Predictors  of  extended  hospital  stay 

Some  payers  will  cover  occasional  hospitalisations  of 
limited  duration  (eg,  S5  days)  for  hospice  patients,  for 
the  purposes  of  respite  care  or  medical  attention  to  a 
condition  which  is  not  the  terminal  illness.1  11 
Therefore,  we  used  a  multivariate  logistic  regression 
model  to  examine  the  dichotomous  outcome  of  long 
(>6  days)  vs  short  (£5  days)  hospital  stays  among  survi¬ 
vors  (table  4).  We  found  that  patients  <64  years  of  age 
were  almost  twice  as  likely  to  experience  a  long  hospital 
stay  (OR=1.89,  95%  Cl  1.08  to  3.33,  p=0.03)  compared 
with  those  aged  75—84  years.  Gender  and  race/ethnicity 
did  not  predict  the  length  of  hospital  stay.  A  total  of  18 
clinical  conditions  were  included  in  the  model;  only  3 
conditions  were  significantly  associated  with  a  longer 
length  of  stay:  cancer  (OR=1.80,  95%  Cl  1.15  to  2.79, 


Table  4  Predictors  of  a  long  (S6  days)  vs  short  (<5  days) 
hospital  stay  among  hospice  patients  who  survived  to 
discharge  (n=1457):  multivariate  logistic  regression  results 

Patient  predictors 

OR  (95%  Cl)  (p  value) 

Age,  years 

<64 

1.89  (1.08  to  3.33)  (0.03) 

65-74 

1.65  (0.96  to  2.85)  (0.07) 

75-84 

1.00  (referent) 

85-94 

0.91  (0.55  to  1.49)  (0.70) 

95+ 

1.16  (0.51  to  2.62)  (0.73) 

Gender 

Male 

1.00  (referent) 

Female 

1.02  (0.71  to  1.46)  (0.94) 

Race/ethnicity 

White 

1.00  (referent) 

Hispanic 

1.13  (0.72  to  1.76)  (0.60) 

Black 

1.22  (0.65  to  2.28)  (0.53) 

Other 

1.08  (0.31  to  3.73)  (0.91) 

Payer 

Medicare/Medicaid 

4.87  (2.50  to  9.51)  (<0.000 1) 

Private  insurance 

1.00  (referent) 

Tricare/Federal 

2.71  (1.50  to  4.89)  (0.001) 

Other 

0.86  (0.42  to  1.76)  (0.68) 

Clinical  diagnoses*  (present  vs  absent) 

Cancer 

1.80  (1.15  to  2.79)  (0.01) 

Heart  failure 

1.65  (1.00  to  2.70)  (0.047) 

Stroke 

1.81  (1.07  to  3.07)  (0.027) 

Psychosis 

1.52  (0.99  to  2.34)  (0.059) 

Discharge  destination 

Hospice 

1.00  (referent) 

Healthcare  facility 

4.67  (2.94  to  7.41)  (0.0001) 

Home 

2.61  (1.49  to  4.57)  (0.0008) 

*The  following  diagnoses  were 

also  included  in  the  model  and  all 

were  insignificant:  chronic  obstructive  pulmonary  disease. 

ischaemic  heart  disease,  pneumonia/influenza,  HIV,  hypertensive 

disease,  non-ischaemic  heart 

disease,  any  fracture,  complication 

of  medical  devices,  respiratory  failure,  drug/alcohol  dependence. 

sepsis,  acute  renal  failure,  end-stage  renal  disease  and  senility. 

p=0.01),  heart  failure  (OR=1.65,  95%  Cl  1.00  to  2.70, 
p =0.047)  and  stroke  (OR=1.81,  95%  Cl  1.07  to  3.07, 
p=0.027).  Patients  with  psychosis  were  52%  more  likely 
to  have  a  long  length  of  stay  compared  with  patients 
without  psychosis,  with  borderline  significance 
(OR=1.52,  95%  Cl  0.99  to  2.34,  p=0.059). 

Finally,  consistent  with  the  survival  analysis  results 
shown  in  table  3,  the  strongest  predictors  of  long  versus 
short  stay  were  payer  and  discharge  destination. 
Compared  with  patients  insured  privately,  extended  hos¬ 
pital  stays  were  almost  five  times  more  likely  among 
those  with  Medicare  or  Medicaid  (OR=4.87,  95%  Cl 

2.50  to  9.51,  p<0.0001),  and  almost  three  times  more 
likely  among  those  insured  by  Tricare  (OR=2.71,  95%  Cl 

1.50  to  4.89,  p=0.001).  Patients  who  did  not  return  to 
hospice  were  several  times  more  likely  to  have  a  long 
hospital  stay  compared  with  those  who  did  return  to 
hospice  (discharge  to  the  healthcare  facility  OR=4.67, 
95%  Cl  2.94  to  7.41,  p<0.0001;  discharge  home 
OR=2.61,  95%  Cl  1.49  to  4.57,  p=0.0008). 
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Financial  charges 

The  median  hospital  charge  for  patients  admitted  from 
hospice  was  $3916  (table  5).  Reflecting  differences  in 
the  length  of  stay,  this  value  varied  considerably  by  dis¬ 
charge  status,  from  $3424  for  patients  who  died  to 
$13  293  for  patients  who  were  discharged  home.  The 
most  frequent  charge  centre  was  pharmacy,  with  91.6% 
of  patients  having  a  non-zero  charge,  followed  by  room 
and  board  (84.4%  of  patients),  medical/surgical  sup¬ 
plies  (58.9%),  respiratory  services  and  tests  (29.4%), 
intensive  care  unit  (1CU,  16.8%),  laboratory  (18.7%), 
radiology/imaging  (10.0%)  and  emergency  department 
(5.6%).  Low  usage  levels  for  laboratory  and  radiology 
are  consistent  with  the  intent  of  exclusion  of 
curative-intent  treatment  for  hospice  patients.  Patients 
who  did  not  return  to  hospice  were  much  more  likely  to 
have  laboratory  charges  (57.7  —71.0%)  and  radiology/ 
imaging  charges  (40.3—  44.2%).  ICU  usage  and  charges 
were  highest  among  patients  discharged  to  a  healthcare 
facility  and  lowest  among  patients  who  died. 

In  summary,  total  charges  for  hospice  patients  admit¬ 
ted  to  hospitals  in  Florida  during  our  30-month  study 
period  exceeded  $25  million  (table  4).  Hospice  patients 
who  appeared  to  be  exiting  hospice  care  were  10% 
(287/2674)  of  patients  by  number  but  incurred  32% 
($8  021  013/$25  265  839)  of  the  financial  charges  due 
to  longer  lengths  of  stay,  a  greater  number  of  proce¬ 
dures  and  a  greater  likelihood  of  ICU  use. 

DISCUSSION 

This  is  one  of  the  largest  studies  to  date  of  a  neglected 
aspect  of  the  experience  of  dying  patients  who  chose  to 
enrol  in  hospice.  We  capitalised  on  a  new  data  item  added 
in  2010  to  Florida’s  mandatory  hospital  discharge  reporting 
system  which  identified  patients  who  were  admitted  to  the 
hospital  from  a  hospice  programme  (either  home  or  facility- 
based).  Our  study  revealed  that  46%  of  patients  admitted  to 
the  hospital  from  hospice  died  before  discharge  and 
incurred  over  $8  million  in  charges.  Most  of  these  patients 
died  on  the  day  of  admission  or  within  1—2  days  after  admis¬ 
sion  and  did  not  receive  life-saving  procedures,  although 
18%  were  admitted  to  the  ICU.  This  suggests  that  these  hos¬ 
pital  admissions  were  medically  unnecessary,  which  is 
important  given  that  hospitals  are  not  the  preferred  place  of 
death  for  many  patients.  Research  carried  out  on  patients’ 
preferences  regarding  place  of  death  has  shown  that  in  an 
idealised  trajectory  of  hospice  care  patients  want  to  die  at 
home  or  in  a  hospice  facility  with  minimal  medical  interven¬ 
tion  prior  to  death.1-  However,  for  some  patients  and  care¬ 
givers,  hospital  admission  in  the  final  hours  or  days  of  life 
may  be  preferred.  For  future  hospice  research,  an  important 
patient-centred  ‘outcome’  measure  may  be  whether  death 
occurred  in  the  place  and  context  desired  by  the  patient 
and  family. 

A  second  group  of  patients  admitted  to  the  hospital 
from  hospice  care  returned  to  hospice  after  their  stay  in 
the  hospital.  Many  private  insurance  companies  as  well 


as  TriCare  and  Medicare  make  allowances  for  ‘respite 
care’  hospital  stays  of  up  to  5  days’  duration  in  order  to 
give  patients’  caregivers  a  short  break.  Respite  care  is 
consistent  with  the  top  two  admission  diagnoses  for  this 
group  (cancer  and  palliative  care).  However,  other 
admission  diagnoses  (eg,  stroke,  respiratory  failure/ 
symptoms,  sepsis  and  psychosis)  suggest  acute  illness  epi¬ 
sodes  that  home  or  hospice  facility  staff  were  unprepared 
to  cope  with.  Although  there  is  a  substantial  body  of 
research  on  caregiver  stress  and  burnout,  this  is  gener¬ 
ally  framed  as  a  possible  threat  to  the  health  of  the  care¬ 
givers  themselves  rather  than  a  possible  reason  for 
hospitalisation  from  hospice.1'-1'  Additional  research  on 
a  possible  link  between  the  preparedness  of  home  and 
hospice  facility  staff  and  acute  illness  episodes  resulting 
in  hospitalisation  is  warranted. 

Importantly,  two  groups  of  hospice  patients  in  our 
study — those  discharged  to  a  healthcare  facility  and 
those  discharged  home — were  patients  whose  admission 
to  the  hospital  most  likely  denoted  a  rejection  of 
hospice  care.  This  hypothesis  is  consistent  with  our 
observations  that  these  two  groups  were  more  likely  to 
have  a  stay  longer  than  5  days,  had  higher  rates  of  pro¬ 
cedure  use,  and  incurred  median  financial  charges  that 
were  much  higher  than  those  who  died  or  returned  to 
hospice.  Although  this  was  a  small  group  of  patients, 
further  study  on  the  phenomenon  of  terminally  ill 
patients  exiting  hospice  treatment  is  needed. 

Study  limitations 

Data  about  the  total  number  of  hospice  patients  in  the 
state  of  Florida  during  our  study  period  were  not  readily 
available.  Hospice  care  is  not  subject  to  regular  surveil¬ 
lance,  and  the  large  number  of  relatively  small  providers 
of  hospice  care  makes  ascertainment  of  denominator 
estimates  difficult.  On  the  basis  of  a  recent  report  from 
a  professional  organisation, 16  we  roughly  estimate  that 
there  120  000  hospice  patients  annually  during  our 
study  period  in  Florida.  This  would  translate  to  about 
1%  of  hospice  patients  being  admitted  to  hospital. 
However,  while  hospital  discharge  data  systems  have 
been  shown  to  be  reasonably  valid  sources  of  patient 
information17-21  and  are  widely  used,22-2'  we  believe  this 
new  variable  to  be  significantly  under-reported,  and  that 
the  true  number  of  hospice  patients  admitted  was 
higher.  In  particular,  patients  who  are  admitted  via  the 
emergency  department  (as  opposed  to  a  direct  admis¬ 
sion  by  the  physician)  may  be  less  likely  to  have  their 
hospice  status  recorded  in  the  medical  record. 

A  second  limitation  of  our  study  is  that  these  termin¬ 
ally  ill  patients  suffered  from  multiple  chronic  and  acute 
diseases  and  we  did  not  have  access  to  data  which  identi¬ 
fied  the  initial  reason  for  referral  to  hospice.  An  import¬ 
ant  empirical  question  is  whether  the  healthcare 
trajectories  of  patients  dying  from  cancer,  for  example, 
differ  in  significant  ways  from  those  dying  from  congest¬ 
ive  heart  failure  or  COPD. 
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Table  5  Detailed  financial  charges  for  patients  admitted  from  hospice  by  final  discharge  status,  Florida  2010-2012 
Charge  centres  (listed  in  Died  before  Discharged  to  a 


order  of  decreasing 
frequency) 

discharge 

(n=1217) 

Discharged  to 
hospice  (n=1170) 

healthcare  facility 
(n=149) 

Discharged 
home  (n=138) 

Total 

(n=2764) 

Total 

Per  cent  with  any  charge 

100 

100 

100 

100 

100 

Median  ($) 

3424 

3634 

11  133 

13  293 

3916 

99th  centile  ($) 

46  755 

59  606 

310  900 

252  858 

100  431 

Total  for  all  patients  ($) 

8  041  068 

9  203  758 

4  074  703 

3  946  310 

25  265  839 

Pharmacy 

Per  cent  with  any  charge 

86.9 

95.0 

98.0 

97.8 

91.6 

Median  ($) 

434 

562 

1949 

1427 

586 

99th  centile  ($) 

25  727 

17  099 

43  220 

34  186 

24  322 

Room  and  board 

Per  cent  with  any  charge 

83.0 

86.3 

81.9 

82.6 

84.4 

Median  ($) 

1802 

1730 

4290 

3912 

2106 

99th  centile  ($) 

20  400 

16  843 

31  898 

49  462 

22  185 

Medical/surgical  supplies 

Per  cent  with  any  charge 

48.7 

67.4 

68.5 

66.7 

58.9 

Median  ($) 

484 

484 

1,167 

548 

525 

99th  centile  ($) 

9942 

8476 

17  892 

193  544 

15  048 

Respiratory  services/tests 

Per  cent  with  any  charge 

23.0 

33.9 

35.6 

40.6 

29.4 

Median  ($) 

522 

585 

851 

548 

571 

99th  centile  ($) 

16  240 

7174 

78  726 

16  050 

16  240 

Intensive  care  unit 

Per  cent  with  any  charge 

17.9 

14.4 

24.8 

19.6 

16.8 

Median  ($) 

3071 

2831 

5542 

1915 

2990 

99th  centile  ($) 

16  276 

38  300 

62  350 

29  505 

32  505 

Laboratory 

Per  cent  with  any  charge 

6.9 

19.9 

57.7 

71.0 

18.7 

Median  ($) 

506 

1041 

3337 

1915 

1368 

99th  centile  ($) 

62  577 

37  357 

41  047 

35  428 

37  357 

Radiology/imaging 

Per  cent  with  any  charge 

2.7 

9.7 

40.3 

44.2 

10.0 

Median  ($) 

1728 

1475 

3135 

2310 

2018 

99th  centile  ($) 

61  895 

22  079 

61  070 

43  598 

48  922 

Emergency  department 

Per  cent  with  any  charge 

1.9 

4.6 

24.2 

26.1 

5.6 

Median  ($) 

1999 

1886 

1647 

1786 

1851 

99th  centile  ($) 

5089 

6134 

7189 

5130 

6134 

A  final  limitation  of  this  study  is  the  lack  of  informa¬ 
tion  available  about  patient  and  family/caregiver 
decision-making  and  preferences  in  relation  to  hospice 
care,  hospital  admission,  ICU  admission  and  use  of 
interventions  such  as  mechanical  ventilation  and  blood 
transfusions.  For  the  patients  who  died,  it  is  unknown 
whether  dying  in  the  hospital  (often  shortly  after  admis¬ 
sion)  was  in  accord  with  patients’  and  caregivers’  wishes, 
or  represented  last-minute  failures  of  hospice  care  to 
shield  patients  from  unwarranted  medical  intervention. 

Conclusions  and  future  directions 

Our  results  and  those  of  previous  studies”-1  raise  difficult 
and  significant  questions  about  the  ethics  and  pragmatics 
of  end-of-life  medical  care  and  the  intentions  and  scope  of 
hospice  care.  Commonly  used  templates  for  patient 
advance  directive  documents,  for  example,  5  Wishes,2'1 


include  lists  of  ‘life-support  treatment’  that  patients  may 
want  to  avoid,  such  as  cardiopulmonary  resuscitation, 
mechanical  ventilation,  tube  feeding,  major  surgery,  blood 
transfusions,  dialysis  and  antibiotics.  In  our  study,  the  use 
of  procedures  was  very  low,  except  among  patients  who 
survived  and  did  not  return  to  hospice.  In  a  report  on 
hospice  patients  with  hip  fracture,7  83%  received  hip 
surgery  and  consequently  had  improved  survival  com¬ 
pared  with  those  who  did  not  undergo  surgery.  Patients 
dying  of  a  terminal  disease  (eg,  cancer)  may  experience 
painful  and/or  life-threatening  acute  illness  events  that 
are  completely  unrelated  in  pathology  (eg,  stroke  or  hip 
fracture),  and  which  were  not  directly  addressed  in 
advance  directives  or  hospice  care  plans.  Patients,  care¬ 
givers  and  healthcare  providers  may  all  find  themselves 
uncertain  about  the  most  ethical  and  compassionate 
course  of  action  in  these  situations. 
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We  would  advocate  inclusion  of  a  simple  hospice  indica¬ 
tor  in  additional  clinical  registries  and  hospital  reporting 
systems,  as  this  would  be  a  low-cost  means  of  creating  add¬ 
itional  data  resources  for  exploring  hospice  patient  health¬ 
care  trajectories.  Future  studies  should  investigate  variations 
in  hospice  patient  healthcare  trajectories  by  cause  of  ter¬ 
minal  illness,  as  well  as  incorporate  patient  and  caregiver 
needs  and  preferences.  An  ideal  study  design  would  pro¬ 
spectively  link  subjective  patient-centred  data  (eg,  advance 
directive  content,  do  not  resuscitate,  patient-reported  values 
and  preferences)  and  objective  healthcare  encounter  data 
(eg,  emergency  room  visits,  inpatient  stays  and  outpatient 
care). 
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